Building Resilient Automated UI Tests for Cloud Applications using Scripless Technonologies

Editor's Notes

• #2 Good morning. Thank you for coming. My name is Denis. It happened that more than 15 years I am working on tools for developers and testers. At different times it were compilers, and IDEs, tools for load and functional testing. At present moment I am focused on UI test automation.At Inflectra we develop Rapise – it is an IDE and execution environment for automated testing of web, desktop and mobile applications.
• #3 Today I would like to share with you experience we’ve got in area of UI test automation for cloud based applications. Let me describe where it comes from. We frequently help our clients to solve complex tasks in testing process setup. In many cases they are big companies using software to support their internal business processes. These are solutions for financial sector, ERP, healthcare, for example blood banks.
• #4 Customers ask for help because their software solutions are complex, implemented using different technologies, frameworks, programming languages and custom components. As a rule such clients are pretty closed and do not provide access to their systems. So today I’ll speak about a real-life experience, but using a demo example. We created it to let everyone be able to download, install and execute.Our example consists of a publicly available application, testing tool and test sources.
• #5 We will talk about testing of Outlook plugin for Office 365. Outlook is a mail client from Micrsoft, Office 365 is a cloud based version of Microsoft Office. The plugin is named TextMiner, it scans email body for signatures and automatically extracts information about name, address, job title, company name, and so on. Outlook plugin is a web application that can work in a browser and in desktop version of Outlook on Windows and Mac.
• #6 Why develop automated UI tests in this case? We are familiar with the test pyramid and know that this type of testing is the most complex and expensive. Is there a way to save resources? Is there an alternative? One can test backend with unit tests. Integration tests can throw email body into backend using CURL. This is simple. The question is what to do with the frontend, with user interface?Though user interface is pretty simple there is one “But”.
• #7 The plugin works in external environment which may change any time. What worked a minute ago may just stop working. Consequences of unexpected and unpredicted updates from Microsoft side can be different. Let’s list a few of them. Plugin load error Plugin display issues Plugin functional issues Moreover such problems may depend on type of a client and it’s geographical location. Reasons of such behavior are hidden in cloud nature of such a service as Office 365. Different accounts are attached to different versions/builds of the application. Probably updates are applied to trial accounts first and in selected regions. For example users from Europe may experience issues with plugin loading and at the same time clients from North America may be just fine. Also accounts created at different times may be attached to different builds of Office 365 server side components. === Client side issues are not visible to us until reported
• #8 So we have conditions when we do not control situation 100%. We are in the cloud and it can be updated anytime. In the case of desktop applications we at least have an opportunity to follow update schedule of the operating system and timely check our application for compatibility on Virtual Machines with specific configuration. In a cloud we do not have an opportunity to control version of the software we are working with. Such situation is not always the case. For example, in commercial cloud systems supporting manufacturing and sales where software failure leads to tangible financial losses, vendors warn about updates beforehand and give a chance to adapt. Unfortunately Office 365 is not the case. It is a bit of a paradox. Office 365 team makes a better system with each update, but it leads to problems for plugin vendors. Speed of reaction and ability to extinguish the fire is very important in such a case. And it is very desirable to detect problems before users. Thus a minimal set of automated UI tests is required. It must check operability of the plugin in desktop versions of Outlook and different browsers. Also the test coverage must be executed constantly in monitoring mode, for different types of accounts (paid, trial), in different geographical zones, to detect problems as early as possible and react appropriately. The question to experts from the auditory. Would you develop automated UI tests in such a case? If not, what is the alternative? If yes, what tools would you use? [interactive session goes here, approximately 3-5 minutes]
• #9  So let; revisit the title of this talk - Building resilient automated tests for cloud-based applications using scriptless technologies.
• #10  Resiliency is about two things: Resistance to changes in application Reuse – makes easier to update Elastic locators – no updates required for small changes in the app Stability of execution Private browsing mode Wait functions (everybody knows) Physical click
• #11 Therefore UI element locator does not depend on the container. In all cases it is the same XPATH for a given element. To get UI element on desktop one needs to attach to embedded browser in Outlook (it is Internet Explorer on Windows), and then use XPATH locator. For a regular browser use XPATH of the frame, where the plugin is loaded, and then XPATH of the element. This enables to build the test system in such a way that test recorded for browser is suitable to run on desktop without any changes.
• #12 This way the customer saved time and resources required for development and support of tests. Additional economy was achieved with ability to record user actions, locators are calculated automatically in this case (though some postprocessing may be required).
• #13 Just a few auxiliary functions were implemented separately for desktop and browsers. For example, navigation to a particular email in a mailbox. Email loading into the system was implemented using SOAP client built into Rapise.
• #15 - Collapsed view, div overlaps
• #17 Frequently clients ask that they could use non-programmers for creation and modification of tests. It happens for several reasons. It can be people coming from manual testing. Or analytics, managers, who are experts in a domain (e.g. launching atmospheric probes) but do not write code. In such cases the team may consist of testers-programmers who deal with locators, user action emulation (for them software is mostly a set of forms and elements) and people who implement test cases and work at a higher level of abstraction. For testers of second type Rapise offers capability to create tests using spreadsheets. Examples of such spreadsheets you see on the screen. The first example is calling a scenario (MyLogin) implemented by a programmer. The second example is filling a form (registering a new book in a library).
• #18 Frequently clients ask that they could use non-programmers for creation and modification of tests. It happens for several reasons. It can be people coming from manual testing. Or analytics, managers, who are experts in a domain (e.g. launching atmospheric probes) but do not write code. In such cases the team may consist of testers-programmers who deal with locators, user action emulation (for them software is mostly a set of forms and elements) and people who implement test cases and work at a higher level of abstraction. For testers of second type Rapise offers capability to create tests using spreadsheets. Examples of such spreadsheets you see on the screen. The first example is calling a scenario (MyLogin) implemented by a programmer. The second example is filling a form (registering a new book in a library).
• #19 Frequently clients ask that they could use non-programmers for creation and modification of tests. It happens for several reasons. It can be people coming from manual testing. Or analytics, managers, who are experts in a domain (e.g. launching atmospheric probes) but do not write code. In such cases the team may consist of testers-programmers who deal with locators, user action emulation (for them software is mostly a set of forms and elements) and people who implement test cases and work at a higher level of abstraction. For testers of second type Rapise offers capability to create tests using spreadsheets. Examples of such spreadsheets you see on the screen. The first example is calling a scenario (MyLogin) implemented by a programmer. The second example is filling a form (registering a new book in a library).
• #20 At present moment I am focused on UI test automation.At Inflectra we develop Rapise – it is an IDE and execution environment for automated testing of web, desktop and mobile applications. The product has success on international markets, but until today was not presented in Russia. We think that it is time to close the gap and see the outcome.
• #21 Thank you for attention. As I promised here is the link to the page where you can get Rapise for free. The link will be active for two weeks. And link to GitHub repository with demo framework source code. If you want to write me directly – use the email.We still have time for questions and answers. Please go ahead.