Downloaded 17 times
Uploaded byadamcarmi
Advanced automated visual testing with Selenium
The document discusses the importance of automated visual testing in software development, particularly in relation to various web browsers, devices, and screen resolutions. It outlines the methods, tools, and technologies involved in conducting these tests, including the use of image comparison engines and the role of different UI testing tools. The presentation emphasizes the necessity for fast feedback and high-quality assurance as release cycles continue to shorten in the industry.
Advanced automated visual testing with Selenium
- 1. Adam Carmi (@carmiadam) Co-Founderand VP R&D at Applitools ADVANCED AUTOMATED VISUAL TESTING WITH SELENIUM
- 2. YOU CAN ANDSHOULD AUTOMATE YOUR VISUAL TESTS!
- 4. AGENDA Why automatedvisual testing? Tools & Technology Where does it fit? Q & A
- 5. WHAT IS VISUALTESTING? A quality assurance activity aimed to verify that a Graphical User Interface appears correctly to users
- 10. WHY AUTOMATE? The testmatrix is too big to cover manually Web browsers Devices Operating systems Screen resolutions Responsive design L10n 3rd Party upgrades
- 11. WHY AUTOMATE? Mobile appsquality is critical Harder to roll back changes Can’t push daily Updates take battery and data Higher quality bar
- 12. WHY AUTOMATE? Release cycleskeep getting shorter
- 13. AGENDA Why automatedvisual testing? Tools & Technology Where does it fit? Q & A
- 14. HOW DOES ITWORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 15.
- 16. HOW DOES ITWORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 24. AND MORE… 1pixel offsets in element positioning Dynamic content Moving elements Images of different size Performance
- 26.
- 27. ImageMagick A powerfulcommand line tool for image processing. Fuzzing is used to eliminate slight color differences An error ratio is usually used to determine a match $ compare –metric AE –fuzz 5% img1.png img2.png diff.png 2246
- 28. THE JAVASCRIPT ENGINES Resemble.js Pixel by pixel + error ratio + anti-aliasing http://huddle.github.io/Resemble.js Blink-Diff Pixel by pixel + error ratio + anti-aliasing + perceptual color distance computation https://github.com/yahoo/blink-diff Looks-Same Pixel by pixel + perceptual color distance computation + ignore Caret https://github.com/gemini-testing/looks-same
- 29. APPLITOOLS EYES Handlesanti-aliasing, pixel offsets, color similarity, and image scaling Dynamic and moving content Compare images of different sizes No error ratio configuration required Validates full UI pages Fast! Supports layout / structure matching
- 30.
- 31. HOW DOES ITWORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 32. QUICK FEEDBACK TOOLS MOTIVATION Get fast feedback on code changes Run tests locally in the background SETUP Render screenshots with a headless browser PhatomJS, SlimerJS Configuration file driven tests CONS Partial coverage Chrome, IE? Old browser layout engine versions Limited navigation
- 33. VISUAL COVERAGE TOOLS MOTIVATION Verify that the app / site looks right in all real execution environments Test as many UI states as possible Execute many tests in parallel SETUP Render screenshots using real browsers on a variety of operating systems and devices in parallel WebDriver, Grid Code driven tests WebDriver, DSLCONS Requires test infrastructure Implement and maintain test code
- 34. SELENIUM BASED TOOLS CODE/ SCRIPT ● Needle (Python WD) ● WebDriverCss (JS WebDriverIO) ● Gemini (JS DSL) ● Selenium Visual Diff (Java WD) ● VisualCeption (PHP CodeCeption) ● Pix-Diff (JS Protractor) ● Shoov (JS WebDriverIO) ● Vizregress (.NET WD) ● Rspec Page Regression (Capibara) ● Applitools Eyes (All WD + Appium) ● Fighting Layout Bugs (Java WebDriver) ● Huxley (Record Playback) CONFIGURATION ● Viff ● CSS Visual Test ● GreenOnion ● Wraith-Selenium
- 35. HOW DOES ITWORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 36. REPORT DIFFERENCES As fileson the file system WebDriverCSS
- 37. REPORT DIFFERENCES As agallery Selenium Visual Diff
- 38. HOW DOES ITWORK? Drive the AUT and take screenshots Compare screenshots with baseline images Report differences Update the baseline
- 39.
- 41.
- 42. AGENDA Why automatedvisual testing? Tools & Technology Where does it fit? Q & A
- 43. WHERE DOES ITFIT? • Component s • Code review • Developers • Designers • QA Visual testing of frontend components by frontend developers
- 44. WHERE DOES ITFIT? • Pages • Page sections • Developers • Designers • QA • Others Full or partial validation of application screens A collaboration tool
- 45. WHERE DOES ITFIT? • Staging vs. Production • Ops • QA Validate your staging deployment using your production deployment as a baseline
- 46. WHERE DOES ITFIT? • Monitoring • Ops • QA No missing resources in production No corruption due to 3rd party data No corruption due to browser / OS upgrades
- 47. Adam Carmi (@carmiadam) Co-Founderand VP R&D at Applitools QUESTIONS?