Download to read offline
Uploaded bySriram Angajala
Automated visual-regression-testing
This document discusses visual regression testing, which compares the visual output of software to detect changes. It introduces the Antenna House Regression Testing System (AHRTS), a tool that automatically compares PDF output documents on a pixel-by-pixel level to test for regressions in new releases of Antenna House Formatter software. AHRTS addresses challenges with manual visual regression testing by offering high-speed performance on large document sets and generating detailed reports on any differences found. The automated approach significantly reduces testing time and effort while improving accuracy and reliability over manual methods.
Automated visual-regression-testing
- 1. A Visual Comparison Approachto Automated Regression Testing (PDF to PDF Compare)
- 2. Regression Testing • Typeof software testing that seeks to uncover new bugs in existing functional areas of a system after changes have been made to them • Ensure that a change has not introduced new faults and that the intended results are still being produced VisualRegressionTesting 2
- 3. Visual Regression Testing •Compares visual output • New file = or ≠ reference file • Different underlying code can produce the same output • Testing is necessary to see if anything changed • Produce wanted results • Produce unwanted results • Break something that was working fine before • Involves testing either • A subset of documents • A large collection of documents (preferred) VisualRegressionTesting 3
- 4. The Antenna HouseProblem • The need to regression test new releases of Antenna House Formatter • At least 8 maintenance releases a year • Regression Testing (Visual Method) • Has been done manually by visually comparing output of old and new files • Time consuming • Prone to missing minor (but important) differences • Can lead to delays in product releases • Done only on candidate release versions • Increase in support calls due to missed errors VisualRegressionTesting 4
- 5. Going Automated Challenges •Compareoutputs visually, not the underlying code •High speed performance •Support for large collection of documents •Method to generate meaningful reports VisualRegressionTesting 5
- 6. What is AHRTS? •Java application • Automated visual comparison tool • Precision pixel-by-pixel comparison of output documents • Scalable & fast regression testing • Regression testing of new releases of AH Formatter • An automated system for comparing versions of rendering software or rendered output • Beyond regression testing • Pre-production system check • Stylesheet development • Installation validation • System(s) certification VisualRegressionTesting 6
- 7. Homogenize Properties Compare Homogenize Properties Is the content thesame? PDF2PDF Select PDF documents to be compared PDF 1 PDF 2 Extract PDF code as character strings Extract PDF code as character strings Return XML Report Page No Difference Yes How does it work? VisualRegressionTesting 7
- 8. Are pages the same? PDF1 PDF 2 Compare Pixel by Pixel Page by Page Convert to Bitmap Convert to Bitmap Return XML Report Page Difference No Create Composite Image of Differences No Return XML Report Page No Difference Yes Format to PDF using XSL-FO How does it work? VisualRegressionTesting 8
- 9. Benefits • Automates regressiontesting • High speed performance • 90%+ reduction in human effort • Handles large document comparisons • Significantly more accurate and reliable • Able to catch more subtle differences • Able to test more often, fewer problems results in overall better product • Fewer support issues VisualRegressionTesting 9
- 10. Conclusion Beyond regression testing,a PDF to PDF comparison tool like Antenna House Regression Testing System, can have a significant impact on improving software as it may play many roles in the development cycle for any system generating visual outputs. VisualRegressionTesting 10
- 11. Thank you! For moreinformation: www.antennahouse.com celina@antennahouse.com
Editor's Notes
- #7 While it was originally designed to be used for AH Formatter, we have found that it can test a PDF of any source, any rendering software, and we have been using to test other PDF products that we provide.