Download to read offline
Exploratory Testing and Automated Testing: Create a Healthy Relationship
- 1. T24 Special Topics 10/6/16 15:00 Exploratory Testing and Automated Testing: Create a Healthy Relationship Presented by: Dawn Jardine Razorfish Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐-‐-‐268-‐-‐-‐8770 ·∙·∙ 904-‐-‐-‐278-‐-‐-‐0524 -‐ info@techwell.com -‐ http://www.starwest.techwell.com/
- 2. Dawn Jardine An associate quality assurance manager for Razorfish in Toronto, Dawn Jardine has worked in several roles in her career. After years in the teaching profession, Dawn changed direction and entered the software field as a front-‐end web developer for an eLearning software company. Dawn expanded her skills and expertise to include instructional design, eventually joining the QA testing team. With more than fifteen years of experience, she has participated in the launch of several eLearning and eCommerce projects, including Canadian Police Knowledge Network, Canadian Tire Corporation, Walmart Canada, and American Express Membership Rewards. Passionate about quality process and women in technology, Dawn can be found on Twitter @dawnjardine.
- 3. 9/21/16 1 Exploratory & Automation Testing: Createa Healthy Relationship Dawn Jardine 2 Introduction
- 4. 9/21/16 2 3 Toronto, Canada 4 1 2 3 4 The Roleof Test Automation Exploratory Testing and Its Role The Mutual Relationship How We Make it Work Agenda
- 5. 9/21/16 3 5 The Role of TestAutomation 6 Why use automation?
- 6. 9/21/16 4 7 Automation can providebetter coverage • for a large code base • when a live site is in maintenance mode • when it supplements manual testing 8 Automation can find defects earlier • finds defects introduced during sprint development • supports process of continuous test-and-fix in lower life cycle • reduces the cost of rework
- 7. 9/21/16 5 9 Automation speeds deliveryto client • continuous integration and delivery during sprint • achieve 100% regression test case execution • release small and release often 10 When do you implement automation?
- 8. 9/21/16 6 11 The decision touse automation • current code base is growing • addition of new features and enhancements are increasing • completion time of manual test case execution is expanding • releases are occurring frequently 12 What do you automate?
- 9. 9/21/16 7 13 Candidates for automatedtests • repeated tests within sprint development cycles • regression tests that are executed pre-release • sanity tests that are being performed during releases to production 14 How do you implement automation?
- 10. 9/21/16 8 15 Planning for automation • choose a valuable platform • involve your current QA team • hire QA developer(s) • keep the process agile 16 Exploratory Testing and its Role
- 11. 9/21/16 9 17 Where does exploratorytesting fit? 18 “Insanity is doing the same thing over and over again and expecting different results” ~Albert Einstein
- 12. 9/21/16 10 19 James Bach,Exploratory Testing Explained “An interactive process of simultaneous learning, test design, and test execution.” 20 Does the QA team need specialized skills? • experience is an asset but not a requirement • critical thinking and problem solving • autonomy and independence • creativity
- 13. 9/21/16 11 21 Can you “explore”if you already know the way? • YES! • it’s not User Acceptance testing • there are many ways of navigating from Point A to Point B 22 The Mutual Relationship
- 14. 9/21/16 12 23 “Assumptions are the termitesof relationships” ~Henry Winkler 24 Build on the strengths of your team • trust • communication and collaboration • empowerment • confidence
- 15. 9/21/16 13 25 How We Makeit Work The challenges and successes 26 The story • online retail e-commerce solution in production • automation in place for pre-release regression • an established agile process in development and delivery • QA test coverage was still the bottleneck
- 16. 9/21/16 14 27 Management approval for change • respect and value of the QA team • don’t be afraid to innovate! • plan for smarter test coverage 28 Shifting the mindset of the QA team • information sessions about exploratory testing • review of existing manual regression tests • created user stories for high level scenarios
- 17. 9/21/16 15 29 Be creative, beorganized and be brave! • mind-mapping sessions for the high level scenarios • manual and automation team participation • absence of test steps • traceability by recording the results • organized a trial run for the regression cycle 30 Gain stakeholders support • business wants return on investment • transparency is key • continually inform • provide metrics
- 18. 9/21/16 16 31 Sample of metricsfor status reports Manual Exploratory Tests Desktop Browsers OS/Devices Total Desktop Chrome FF IE11 MAC Safari IOS8 Chrome IOS9 Safari IOS8 Safari Android Chrome #of Test Cases 91 9 15 10 12 14 9 12 10 Pass 91 9 15 10 12 14 9 12 10 Fail 0 Blocked 0 0 0 0 0 0 0 0 0 Unexecuted 0 0 0 0 0 0 0 0 0 Executed 91 9 15 10 12 14 9 12 10 Percentage ExecuHon Complete 100% 100% 100% 100% 100% 100% 100% 100% 100% 32 Continued collaboration • ongoing audits of current and future release work • review recorded results from exploratory tests • Repeat, Repeat, Repeat!
- 19. 9/21/16 17 33 Benefits of testingstyle • the QA team’s confidence level grows • the project management can stick to release timelines • the business benefits by staying competitive 34 May 2014 • 261 manually executed step-by-step test cases • No automation in place • 6 days x 4 QAs January 2016 • 49 Exploratory Test cases • 1140 total automated tests covering 4 browsers • 3 days x 2 QAs with focus on Mobile vs Rewards & Results
- 20. 9/21/16 18 35 What’s next? • automatethe mobile web and mobile app regression tests • streamline current exploratory tests • change the format of our sprint functional tests 36 Conclusion
- 21.