Download to read offline
Uploaded bySmartBear
Test Design + Environment Management: Scaling UI Automation for Agile
The document discusses challenges in UI automation and test environment management, particularly focusing on flaky tests and their costly maintenance. It emphasizes the importance of independent tests, efficient identifiers, and the need for modular test designs that integrate GUI and API testing. Additionally, it highlights the benefits of cloud test environments for effective management and scalability in agile testing processes.
Test Design + Environment Management: Scaling UI Automation for Agile
- 1. Test Design +Environment Management: Scaling UI Automation for Agile
- 2.
- 3. 3 Broad automation &test management tools GUI Layer API Layer Developer QA Engineer Manual Tester TestCompleteTestLeft TestServer Ready! API Desktop Mobile Packaged AppsWeb CrossBroswerTesting Selenium/ Appium Target Apps QAComplete Test Management for risk-based analysis Environment Management
- 4. 4 On our platetoday 2 Flaky tests & ways to overcome these problems 3 How to overcome challenges due to frameworks 1 The typical automation process 4 Test Environment management practices
- 5. 5 The process lookssomething like Create tests Run tests Report & Metrics Find bugs 1 2 3 4
- 6. 6 But as yourtests grow you realize Random Is it a bug?? Start ignoring Limited collaboration
- 7. 7 Shift the focuson quality rather than quantity Quality Quantity
- 8. 8 Common reasons forflakiness Expensive to maintainNot Stable Flaky Tests Flaky Framework
- 9. 9 And to dothat set up a process that 1 2 3 Number Time
- 10. 10 Common reasons forflakiness Expensive to maintainNot Stable Flaky Tests Flaky Framework
- 11. 11 Flaky tests resultfrom Expensive to maintainNot Stable Sleeps Stable identifiers lacking Rely on flaky services Interdependent Unreliable tests
- 12. 12 Flaky tests resultfrom Expensive to maintainNot Stable Sleeps Stable identifiers lacking Rely on flaky services Interdependent Unreliable tests
- 13. 13 Things to lookfor identifiers ‐ ID ‐ Classes ‐ CSS ‐ Xpath – Depends on page structure – Won’t change in future – Speed – Exclusive
- 14. 14 Flaky tests resultfrom Expensive to maintainNot Stable Sleeps Stable identifiers lacking Rely on flaky services Interdependent Unreliable tests
- 15. 15 Hardcoded sleeps ordelays Expensive to maintainNot Stable Delay (ms) Sleep (ms) Smarter wait mechanism Wait Untill, jQuery.active == 0
- 16. 16 Flaky tests resultfrom Expensive to maintainNot Stable Sleeps Stable identifiers lacking Rely on flaky services Interdependent Unreliable tests
- 17. 17 Tests need tobe independent Expensive to maintainNot Stable 1 Open the login page 2 Enter username and password 3 4 Multiply two numbers Check the result Two Challenges Not independent Longer than necessary
- 18. 18 Flaky tests resultfrom Expensive to maintainNot Stable Sleeps Stable identifiers lacking Rely on flaky services Interdependent Unreliable tests
- 19. 19 Relying on flakyservices Our App System2 System3
- 20. 20 Common reasons forflakiness Expensive to maintainNot Stable Flaky Tests Flaky Framework
- 21. 21 Not much thoughtinto framework Expensive to maintainNot Stable Login Forgot Password
- 22. 22 Instead separate actionsand control identifiers Expensive to maintainNot Stable Object Class Test Abstract control identification Abstract control actions Test Steps
- 23. 23 So now mytest looks like.. Expensive to maintainNot Stable Abstract identifiers Abstracting actions Test Step
- 24. 24 Infact, don't tryto do everything on the UI UI API Unit $$ $
- 25. 25 Catch bugs earlierand faster Web Mobile Tablet 1 2 3 4
- 26. 26 You can evenreuse these tests across teams API functional test API load test GUI functional test API security test Monitoring Time reduced by 80%
- 27. 27 On our platetoday 2 Flaky tests & framework 3 How to overcome these challenges 1 The typical automation process 4 Test Environment management practices
- 28. 28 Now we aregood at the first half Record & Replay Script QA Engineer/ Manual Tester OSBrowsers Resolutions
- 29. 29 Environment setup oftenneeds manual efforts VMs Speed Redundancy Cost Discrepancies Advanced testing features missing Device LabsLocal Machines 1 2 3 UI Testing Manual Efforts Necessitated 5 4
- 30. 30 For effective environmentmanagement Easy to install Easy to enhance Easy to share Easy to tear down 1 2 3 4
- 31. 31 Cloud test environmentscome handy Example: 800*600, 1680*1050 1920*1440 Example: IE 9, 10, Chrome 50, 55 Firefox 49 Operating System Browser Resolution Example: Windows 10 Windows 8 Windows 7 Cloud Test Environments
- 32. 32 So to recap 1 Followsteps to overcome flaky test and flaky framework 2 Start small and build a modular test design 3 Don’t limit yourself to test automation 3 Integrated GUI and API testing is key Learn more about Environment Manager
- 33.