Downloaded 33 times
Uploaded byWorksoft
Agile-plus-DevOps Testing for Packaged Applications
The document discusses the integration of Agile and DevOps in enterprise software development, emphasizing the importance of continuous testing in the delivery of packaged applications. It outlines trends, key metrics for increasing agility, and the evolution of testing practices, including the shift towards automation and AI-driven solutions. The document also highlights the challenges faced by organizations in adopting these methodologies and the need for a federated testing center of excellence to enhance efficiency and collaboration.
Agile-plus-DevOps Testing for Packaged Applications
- 1. Welcome! Agile-plus-DevOps Testing for Packaged Applications September2018 ©2018 Worksoft, Inc. All trademarks and product names mentioned herein are the property of their respective owners in the United States, Canada and/or other countries. The information contained herein is subject to change without notice. All rights reserved.
- 2. Agenda 3 Part 1 –“Why” • Agile-plus-DevOps in the enterprise • Customer trends • Future outlook Part 2 – “How” • Custom apps vs. packaged apps • Successfully adopting Agile- plus-DevOps for packaged apps
- 3. © 2018 FORRESTER.REPRODUCTION PROHIBITED. September, 2018 Packaged App Testing in the Age Of Agile and DevOps Diego Lo Giudice, VP & Principal Analyst, serving AD&D
- 4. 5© 2018 FORRESTER.REPRODUCTION PROHIBITED. Leading organizations make SW development their core competency! Why ? Digital Matters... Software Makes Digital Successful... Excellence in Delivering Great Business and SW Applications: A Must for everyone !!!..
- 5. 6© 2018 FORRESTER.REPRODUCTION PROHIBITED. End to end agility makes it happen! Agile DevOps Speed, Quality and CX Common Language AGILE Infrastructure Automation Infrastructure: Grip, Security Scaled Infrastructure AGILE Development
- 6. 7© 2018 ForresterResearch, Inc. Reproduction Prohibited Source: Forrester Agile Panel Survey 2017 Base: 232 Agile + DevOps is better than just Agile…. Faster Business value 41% Faster Business value 69% Improved functional quality 47% Improved Functional quality 72% Greater frequency of delivery 75% Improved technical quality 29% Greater frequency of delivery 83% Improved technical quality 72%
- 7. Testing is rightin the middle of the turbulence and becomes continuous too.. Test ReleaseBuild MonitorIdeate/Plan Developers UsersFeedback loop Delivery Pipeline Insights Data/analytics Agile DevOps+ Continuous SW Delivery
- 8. 9© 2018 FORRESTER.REPRODUCTION PROHIBITED. Continuous delivery => Continuous testing
- 9. 10© 2018 FORRESTER.REPRODUCTION PROHIBITED. Agile Is Slowly But Steadily Scaling In Firms This is a Horizontal and Vertical scale Source: The State of Agile 2017: Agile at Scale. Agile + DevOps Slowly But Steadily Reaches Enterprise Scale 2013 2017 2013 2017 25% 75%
- 10. 11© 2018 FORRESTER.REPRODUCTION PROHIBITED. Departmental IT or app dev team for mobile applications development (custom or packaged applications for smartphones or tablets) Departmental IT for internal applications (packaged application software like ERP, CRM, PLM, SCM, SRM, finance or business intelligence) 13% 14% Central IT for internal applications (packaged application software like ERP, CRM, PLM, SCM, SRM, finance or business intelligence) Central IT app dev team for mobile applications development (custom or packaged applications for smartphones or tablets) 36% 19% > 50% What kind of software development is your organization primarily responsible for? Source: The State of Agile 2017: Agile at Scale. Agile + DevOps Slowly But Steadily Reaches Enterprise Scale
- 11. 12© 2018 FORRESTER.REPRODUCTION PROHIBITED. Seven scaling areas of lessons learned: Lights and Shadows But Agile Expert Firms do it all better Source: The State of Agile 2017: Agile at Scale Agile + DevOps Slowly But Steadily Reaches Enterrse Scale
- 12. 13© 2018 FORRESTER.REPRODUCTION PROHIBITED. Key Metrics that increase agility, and Agile Expert firms have higher scores… Source: The State of Agile 2017: Agile at Scale Agile + DevOps Slowly But Steadily Reaches Enterprise Scale
- 13. 14© 2018 FORRESTER.REPRODUCTION PROHIBITED. Continuous Testing, lessons from Expert Firms Source: The State of Agile 2017: Agile at Scale. Agile + DevOps Slowly But Steadily Reaches Enterprise Scale Experts Neophytes
- 14. 15© 2018 FORRESTER.REPRODUCTION PROHIBITED. “In three to five years, the TCoE as we know it will change: shifting to become practice centers” (January 2013) Source: January 15, 2013, “Consistent Performance In Agile Teams Must Include Testing” Forrester report, Diego Lo Giudice From: Centralized To: Federated Packaged Apps: - Regression Testing - End-to-end Testing
- 15. 16© 2018 FORRESTER.REPRODUCTION PROHIBITED. The new state of Testing in Agile+DevOps packaged app delivery Ruthless Automation Shift Left – QA Built in New Organization & Roles Performance testing shifts left SVT automates integration testing Security Testing gets automated Test Data Management Artificial Intelligence/ML
- 16. 17© 2018 FORRESTER.REPRODUCTION PROHIBITED. For automation, technology does matter !......... And for Packaged apps technology requirements can be different SAP, Oracle, Salesforce… CI/CD technology Testing persona Areas of Differentiation to look for API Testing (protocols, etc) More data driven testing SaaS Partner strategy AI / ML
- 17. 18© 2018 FORRESTER.REPRODUCTION PROHIBITED.
- 18. 19© 2018 FORRESTER.REPRODUCTION PROHIBITED.
- 19. 20© 2018 FORRESTER.REPRODUCTION PROHIBITED.
- 20. 21© 2018 FORRESTER.REPRODUCTION PROHIBITED. Can AI improve the way we test? Augment and automate ?
- 21. 22© 2018 FORRESTER.REPRODUCTION PROHIBITED. Most GSIs, new startups and even existing test tool vendors are experimenting new approaches by leveraging ML, DL and NLP. To test smarter and automate less Augment testers with better insights Optimize and generate test cases Increase Automation generation Prevent bugs besides minimize MTTR Optimize test data Automate CX testing ………..and more
- 22. 23© 2018 FORRESTER.REPRODUCTION PROHIBITED. Image source: https://blaesbjerg.com/den-mentale-liste/ If you think that being in the packaged or business apps world you can’t be agile, test better and faster, think more…
- 23. FORRESTER.COM Thank you © 2017FORRESTER. REPRODUCTION PROHIBITED. Diego Lo Giudice +39 340 7710775 dlogiudice@forrester.com @dlogiudice
- 24. How to AdoptAgile-plus- DevOps for Enterprise Applications Chris Kraus, Sr. Director of Product Management
- 25. 11/9/2018© 2017 Worksoft,Inc. Today’s Modern Enterprise Applications. Houses or Cities?
- 26. Release Train forEnterprise Operations Multiple ALM platforms puts the pressure on Continuous Testing Mobile Change ERP Change Web Change CD Continuous Delivery - Release Train DEVELOP GROOM STORIES BACKLOG CODE CHECK INSPRINT STARTS RUN TEST EXECUTION TEST AUTOMATION CHANGE IMPACT ANALYSIS GO LIVE DEVELOP GROOM STORIES BACKLOG CODE CHECK INSPRINT STARTS SolMan Panaya JIRA Project NewWeb JIRA Project NewDevice Multiple systems for code check in Subversion GitHub SAP Transport ERP rollouts in different ALM products More PPM is observed Sales Order to Cash Business process
- 27. No Requirements. No ConsolidatedChange Lists. • Understanding of a process lies with the business • A single process can include 100s of steps • Multiple sprints working in parallel but not on the same board • No easy way to map from requirements > source control > change lists > test management system 28 “We struggle with knowing exactly what to test and exactly are business processes are. With the adoption of the Fiori interface and SuccessFactors, we see things only becoming more complicated.” - SAP Functional Lead Key Considerations
- 28. Consider Agile Testingin the Traditional DevOps World 29 Application Owners / DevTest Pre-Prod Prod Sprint Team “The only thing more dangerous than a developer is a developer conspiring with Security.” ― Gene Kim, The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win
- 29. Key Considerations End-to-End BusinessProcess Testing • Ownership of test creation and maintenance • Test reuse/scalability – ability to feed CI/CT cycles • UI version changes • Compliance and audit requirements • Impact on upstream and downstream system 30 “Our biggest concern is down time. We currently schedule at least 40 hours maintenance a month and we are a 24/7 business. We have too many integrations with SAP today. One of our biggest challenges is testing. Today, we have to get a bunch of people together to build test scripts, then get those people back together in a room to execute tests, this is a big waste of time.” - Business Application Manager
- 30. Modern Apps. ModernCOE. Rise of the New Federated Testing Center of Excellence 31 Plan Prepare/Explore Realize: Build Test Deploy Run Process Owner Sprint Team Federated Testing COE Operations • Business Owner • Portfolio Management (LPM) • Epic Owners • Enterprise Architect • Dev Team • Scrum Master • Product Owner • Test Automation Specialists • Sprint teams work only on small stories and do not see the larger epic • Sprint teams work in dev environments with limited data • End-to-end automation specialists kept as centralized resources • Testing works with pre-production systems testing across full systems with supporting data Enterprise Problems
- 31. The Second Loop:Continuous Testing 32 Realize: Build Test Deploy N+1 Automation Functional experts run capture to document the stories. Once a feature is complete, test automation professionals can finish the automation and schedule into continuous testing cycles. Sprint 1 - 3 Sprint 4 - 6 Unit Tests Unit Tests Functional Test Functional Test Functional Integration Test Sprint 7 - 10 Sprint 11 - 14 Unit Tests Unit Tests Functional Test Functional Test Functional Integration Test
- 32. Scale to ContinuousTesting 33 Plan Prepare/Explore Realize: Build Test Deploy Run Automation Library
- 33.
- 34. DevOps Tool Chain •What technologies will you need • Who will be using • What needs to work together UI Driven • Logging in and out of system • Differences in UIs and version updates • Orchestration and execution of tests on demand at scale 35 “We need to extend ERP testing thru CI/CD so we can release changes at the pace we need to.. Sr. management wants us to be more agile and ensure changes don’t break things upstream or downstream from where that change took place.” - Sr. Mgr. Technical Services Key Considerations
- 35. Tool Chain Complexity– Many Tools Working Together that Need to be Integrated 36 Requirements Management Defect Reporting Test Management Test Data Management Mobile Device Lab Test Orchestration Continuous Integration Continuous Testing Application Lifecycle Management (ALM)
- 36.
- 37. Created a robustCOE as a foundational capability to establish process and framework. Adopted agile methodology and made “automation everyone’s job”. Consolidate high maintenance SAP tests from HP UFT and Turnkey cFactory to Certify. Analyze Capture leveraged to accelerate automation creation for Manhattan. Automation extended to RPA and used for SAP mass configuration. Builds a Foundation for Continuous Testing and RPA GOAL Continuous Testing & Integration and RPA 40% reduction in automation maintenance. 60% reduction in regression test execution. 90% automated in sprint test coverage. Sources: SAP SAPHHIRE NOW 2018 Presentation and SAP Insider Article Fall 2018 “A Testing Center of Excellence (COE) should be geared toward mission- critical applications- focused on your most vital projects. The COE enables execution & shares best practices.” – Nikhil Shah Cardinal Health is a Fortune 100 health care company serving more than 100,000 locations world wide. 85% reduction in mass configuration efforts.
- 38. Related Resources • InfoPage for Capture 2.0: https://www.worksoft.com/products/capture • DEMO: Daily Continuous Testing with Worksoft and Jenkins • DEMO: End-to-End Agile Testing for SAP with Worksoft Certify • ALM Integration Support • HP ALM: https://www.worksoft.com/micro-focus-hp-alm-integrations- support • SAP Solution Manager: https://www.worksoft.com/testing-framework- sap-solution-manager • Worksoft University (Free Online Self-Paced Courses) 39
- 39.