Uploaded byJacky Wu
Agile Qa Framework Jacky Wu
This document outlines an agile QA framework for testing in a scrum environment. It discusses roles like developers, QA and AQA working interchangeably. The framework focuses on a whole team approach with automation, a balanced process, and incremental testing. It provides guidelines for infrastructure, process, defect management and an automation approach using a defects derivative model and business process testing. The goal is to improve quality through a balanced emphasis on prevention, automation and other factors.
Agile Qa Framework Jacky Wu
- 1. Agile QA FrameworkJackyWujackywxd@gmail.com
- 2. Agile ManifestoWe Valuethe items on the left more !
- 3. AgendaGoal of QAFrameworkRoles and Role Interchangeability in ScrumKey success factor: Whole Team ApproachQA Framework and Execution GuidelineInfrastructureProcessAutomation in AgileWhole team and Incremental Approach AutomationDefects Derivative ModelBPT Automation FrameworkWhat’s NextRecap and Q&A
- 4. Goal of QAFramework
- 5. Roles in ScrumDEVModelsystem designKnowledge of product internalsFocus on how it can workQAModel user behaviorDomain knowledgeFocus on how it can go wrongAQAMixed both DEV/QA
- 6. Role Interchangeability inScrumSprint planningSprint ReviewDesign, Coding, DebugUnit testing, code reviewTest cases executionTranslate User Stories into Test CasesAutomation feasible analysisAutomation testing
- 7. Whole Team ApproachWholeteam is responsible for qualityTesters are not quality Police
- 8.
- 9. Execution Guideline: Balanceand Adaptive Process Guidance above Process AbsolutePrevention above detectionAutomation above manualReusable lists above detailed test plansExploratory testing above detailed test scriptsThe Key: Balance and Adaptive
- 10. QA Framework: InfrastructureQualityCenterBuilt-in Agile supportDefect Defect model in QCTesting bedReal WorldPerformance/stress testing bed
- 11. QA Framework: ProcessTestplan management (Quality Center)Iterations and RequirementsMapping user story to test planTest data managementEfficient Test Data Management process benefits manual and automated testing and it directly impacts testing effectivenessObtaining/creating test data could be time consumingDefect managementContinuous IntegrationData collection and reportingDefect trending/Test progress/Automation coverage
- 12. QA Process inSprint (Sample)
- 13. QA Framework: Automationin AgileAutomation is MUST in AgilePurpose of AutomationImprove testing effectiveness and efficiencyUltimate goal: Improve QualityAutomation v.s. ManualMaximize ROIBalance: what can automated, what cannot or should notWhole team approachIncremental approach
- 14. Whole Team ApproachAutomation in ScrumEnd UserCode
- 15. Incremental Approach Automationin SprintCCCCGGGBBC: Component automationG: GUI automationB: BPT automationI: IntegrationIDevelopmentStable
- 16. Automation MeasurementDefine AutomationCoverageCode coverage = UTTest case coverageUnique case (with or without iteration)= Automated test cases/Total test casesRun coverage= Automated test runs/Total test runsRequirement coverageAutomation can find more bugs?
- 17. Defects Derivative ModelCostRequirementsWrongrequirements$PlanCorrect DesignCorrect DesignWrong Design$$DesignCorrect implementationCorrect implementationCorrect implementationWrong implementationCorrect implementationWrong implementation$$$CodingCorrect behaviorsUnexpected behaviorLimitationsKnown bugsUnknown bugTesting$$$$
- 18. Business Process TestingProcessSeparate automation script from test data and business logicLess test cases, many iterations (testing data)Central management: testing plan, testing data, testing resultQCApplicationBeing TestedQTPLoginData-DriverBPT Test CasesBPTBackupBPTTesting dataJob StatusBPT
- 19. BPT Automation FrameworkAutomationDefinecomponentsCreate Function LibrariesCreate Object RepositoriesCreate Business ComponentsCreate Business ComponentsDesign user scenariosSMEDrag Components to create test plan Configure Input/Output parametersAdd cases to test set in Test Lab and Execute
- 20.
- 21.
- 22. Referenceshttp://en.wikipedia.org/wiki/Test_automationhttp://testingeducation.org/BBST/http://www.ncpmi.org/userfiles/File/NCPMI_AE2010_Lawson.pdfhttp://c-spin.net/2010/cspin201001eMids_QA_in_Agile.pdfhttp://www.cigital.com/presentations/Agile%20Automation%20Testing.pdfhttp://www.benchmarkqa.com/pdf/papers_automation_myths.pdfhttp://www.methodsandtools.com/archive/archive.php?id=94Case in point:Microsoft Vistahttp://www.joelonsoftware.com/items/2007/12/03.html
Editor's Notes
- #10 So how to execute the QA framework in Agile context? There are few guidelines, and the key is balance and adaptive. As mentioned previously, every scrum team is unique. Scrum team has to find the best suitable approach based on the actual status of each scrum team.
- #13 So far QA is the major user of QC. This is an example process which involves all roles in Scrums.
- #16 Automation in Sprint also will incremental
- #17 There is no doubt that Automation is important. But how do we know how are we doing? How to measure automation testing? One of the common way is automation coverage.Code coverage: cover how many line of codes and possible code pathTest case coverage: percentage test cases that are automatedRequirement coverage: cover how much percentage of user scenariosOne execution of test case is one test run.How is test coverage defined? Are we measuring test cases against requirements (generally during system testing), or are we measuring test cases against all possible paths taken through the units and components (generally used for unit testing)? In other words, are we looking at unit testing coverage, code coverage, or requirements coverage?Another measurement could be how many bugs are found by automation? So can automation testing find more bugs? In next slide of defect derivative model, we can look into it.
- #18 Which type of defects can be found by Automation?From the cost effective perspective, the best place to use automation is to increase the chance of correct implementation during coding. Example, plain password in the log file. So UT automation, component level automation is very important. These type of automation can help us find problem before it becomes defect.Regression testing also is another type of testing can leverage automation. Actually I believe regression has to use automation.But no matter what, we don’t think today we are over-automated, given the same amount of investment how can we increase automation coverage? The answer is BPT and Data driven.
- #19 So what’s BPT? BPT stands for Business Process Testing. In the end of the this slide, there are more details information about BPT. It also fits into whole team approach and incremental approach.Less test cases; ASBU used to have more than 10 thounds of test case, one of the reason is we mixed test data with our function test case. Sankar and I actually discussed it previously, we all agree we should reduce the number of test cases. And using BPT is one of the effective way to reduce test cases.
- #20 BPT is the automation framework which can leverage both strength of DQA and AQA. It also can free AQA’s hand so AQA can focus on the most important thing: coding, instead of maintain. Once component is turned over to DQA, DQA will take the ownership of automation, create, execute and monitor automation test cases will be handled by DQA. Moving forward, we also expect DQA can pick up some of the tasks from AQA, moving up the bar.Benefits of BPT:Fully leverage strength of AQA and DQAIt fits into Agile development and QA processMaximize Automation ROIKeep track automation usages and provide comprehensive reportsNo additional cost incurred