Downloaded 16 times
Uploaded byDerk-Jan de Grood
Testing fundamentals in a changing world (annotated slides)
The document discusses the evolution of testing fundamentals in a rapidly changing technological landscape, focusing on shifts toward automation, integration, and value-driven business testing. It emphasizes the necessity of frequent product launches and a collaborative approach to testing while highlighting the decline of traditional testing methods. The overarching theme advocates for a transition from project-centric testing to an operational mindset, where integration, stakeholder feedback, and quality assurance are prioritized.
Testing fundamentals in a changing world (annotated slides)
- 1. Testing fundamentals ina changing world Derk-Jan de Grood SIGIST London September 2016 1
- 2. Shifts Releasing x-times aday reduces the need for formal acceptance Shift to Automation and Bug hunting Shift to Business tests Towards value and systems thinking Shift to Non- functional tests From Project to Operations Methodic testing is declining From: BUFD To: JIT
- 3. Aim of thissession 5
- 4. 6 Beyond the T-shape (Tutorial/webinarwith Jan Jaap Cannegieter)
- 5. Beyond the T-shape (Tutorial/webinarwith Jan Jaap Cannegieter)
- 6.
- 7.
- 8.
- 9. CI/CD Assumptions 1 Teams Collaborate Integration is Continue Testsare Automated Deployment is hands-off process No Automation Backlog Clear Acceptance Criteria Feedback loop to improve Testing Frequent Product Launch
- 10. 12 A test report stillhas value YES ! NO ! Releasing x-times a day reduces the need for formal acceptance
- 11. 13 The value ofmy work….
- 12. The testers mind 14 UT,ST, SIT, UAT Big Data Test techniques Documentation New Technologies Metrics IoT Defects Risks Stakeholders Process Stayin’ in Demand
- 13.
- 14. It ain’t doneuntil its accepted 16 Witness Development en Tes ng Tested solu on Witness report Customer
- 15. Witness report: 2outcomes Advice to accept solution • Professional testing • Good process • Good test results Advice to not accept solution • No professional testing • No Good process • No good test results 17
- 16.
- 17. In the realworld Stakeholders need status info 19
- 18. 20 It ain’t doneuntil it’s tested
- 19. Testing a CentralTopic Has the train arrived at the station? 21
- 20.
- 21. 23 It ain’t doneuntil it’s tested
- 22. It ain’t doneuntil it’s integrated 24
- 23.
- 24. Architecture • What arethe business processes? • What are the components? • What are the interfaces? Acceptance criteria • What is the Minimal Viable Product? • What integrations are needed to make it work? Requirements traceability • When are we complete? • How do test results add up to acceptance? 26 Missing
- 25. What should acar minimally do? 27
- 26.
- 27. Effects 29 Witness Development en Tesng Tested solu on Witness report Customer
- 28. 30 Reduction of depencies EarlyFeedback Early Error Detection Shift to Automation
- 29.
- 30. 32 From Silo toTeam effort Methodic testing is declining
- 31. How much documentation? 33 Typeof Test Throw away Regression Compliance Smoke & Monitoring From: BUFD To: JIT
- 32. TBYDWTFI P 34 Shift to Non- functionaltests Annotation: Asking the stakeholders for The Bugs You Don’t Want To Find In Production will define the test you need to do before the code goes live. No doubt stakeholders will mention non functional items like Security, UX and Performance
- 33.
- 34. Ensuring Integration (roughsketch) 36 Organization Component System Service Continuously (in the sprint) Occasionally (e.g. prior to a release) Towards Value and System thinking Annotation: Increasing the system (e.g from Units tot Systems) results in less frequent integration, because it becomes harder to test the integration. This has impact on the time-to-market.
- 35. 37 Shift to Business tests Annotation:Business test verify the business results of the change or new service. Is the reduction in costs, the expected revenue earned?
- 36.
- 37. Agile Test Stategy 39 Whatneeds to be tested Steering committee Dashboard/ Roadmapping Auditing the test work Coaching the testing team members Organizing tests that do not fit the sprint We still need a plan to know were complete There is a need feedback and traceability (comfort & adaptivety) Testing needs be of quality to justify itself Testing needs to add value to hold up an release
- 38. Test Report 40 Does it workas a whole?
- 39. Tell the testingstory Do you know what is tested in the various teams? Do you know how well they did it? Do you have proof? How do you rate the quality? What concerns do you have (combine the bugs and translate to busines impact)? 41
- 40. 42 Please, will youdo some testing for me
- 41.
- 42. Shifts Releasing x-timesa day reduces the need for formal acceptance Shift to Automation and Bug hunting Shift to Business tests Towards value and systems thinking Shift to Non-functional tests From Project to Operations Methodic testing is declining From: BUFD To: JIT
- 43.
- 44. 46 Integrated with development Integratedwith Business Efficient and Complete Traceable Automatic Is perceived to have value
- 45.
- 46. Derk-Jan Valori Coltbaan 4a 3439 NGNIEUWEGEIN The Netherlands • derkjandegrood@valori.nl • +31(0)651807878 • www.valori.nl • @DerkJanDeGrood • http://djdegrood.wordpress.com 48
Editor's Notes
- #8 Webinar/tutorial on Pi shaped tester- there is a lot of choice 42 profiles, but a lot of people lack knowledge…
- #10 But lets focus on testing
- #12 Teams collaborate to deliver each sprint a working product Integration is continue Tests are automated Tests are run from the build server Deployment is hands-off process TDD ensures no backlog in TA Acceptance criteria are clear Feedback loop to create better tests in place Product goes live regularly
- #13 Release advice does it have value? When releasing multiple times a day….. What is the need for a test report / advice TADAY dame claimed 400 times a day a release
- #14 Workshop where we guided a department of IM to define their roles in Agile context…
- #18 Scenario advice to not accept: nobody wants this shouldn’t be a surprise
- #19 The way to avoid the second scenario is by putting into place witness process Set of goals that need to be reached by 4 activities. Extensive reviewing Witnessing is an ongoing phase and is all about transparency and working together. Avoid surprises, mitigate risk, clear understanding on quality0 Audit: De leverancier toont aan dat: checklist van uit te voeren activiteiten, -> paper trail klopt! Bewijslast ligt bij leverancier. Demo: periodieke inzage in kwaliteit van product. Toont test werkzaamheden van afgelopen periode, testscenario’s worden afgespeeld en inzicht wordt gegeven van de belangrijkste bevindingen (wat werkt nog niet). -> Demo rapport! Meetings: Strategisch testoverleg, Finding Meeting, Demo -> Kwaliteitsoverzicht Review: Testplannen, testcases, Testresultaten, Non functional testen.
- #23 Due to the shift from projects to operations, there is no end point, it’s a continue process. Which creates a shift towards Automation and bughunting. Who needs a formal release advice/report Eet your own shit….
- #25 We do not want a screen, we want a feature
- #30 @Split: Technical tests could be: Exploraotry tests, Bughunting, unit test, ST testing, performance testing, interface testing, Business rule testing Business tests could be AO , e2e-chain, Testing in the large Note the Pi-shaped tester: More than one specialism, which one you chose
- #33 Dev does the testing Met a lot of developers want to the the minimal which is good, but do they oversee the total picture Testers might feel misunderstood, or alone in their quest for quality Testers matrix: keep them clicking Method testing seems to vanish Testing is not fixed in the policy but in the DOD Methodic testing is declining Does your testing still have value (in the eye of the beholder?)
- #36 Integration becomes important if we start thinking in featues and business epics rather than system adaptions
- #37 How to ensure Integration We do: CI/CD MBT UT Automated System test Automated e2e test Interface testing Manual Regression testing Integration sprints Other General trend: Increasing the system (e.g from Units tot Systems) results in less frequent integration, because it becomes harder to test the integration. This has impact on the time-to-market and this insight might lead to targeted improvements
- #38 The above strategy has a focus on Acceptance and getting the service, product life Should we do business testing also…. This does not happen that much What hooks do you need in the code to measure this? Is it the next stage in the agile implementation model?
- #39 When we need to take of so much items, wide spread, do we believe it will work out itself, What do we need? Yes a test strategy
- #46 Widespread: Busines involvement, Organisational readyness, Governance, product vision Integrated in the process: erverybody is testing, tests are done throughout the process and a lot of communication is involved; top keep it simple and effective Technincal : Automated, unit tests and intregrated in the build process, Fast (every day evry sprint).