Agile Testing - Challenges

Agile Testing - Challenges

• 1.
  Challenges in AgileTesting
• 2.
  List of Challenges Volume and Speed of requirement change  Over-commitment and Sprint durations  Accidental Broken Code due to frequent builds  Inconsistent and Inadequate Unit Testing  Inadequate Test Coverage  Inadequate Testing for published API  Poor, Changing and Missing Test Oracles  Detecting defects early, When they’re easier and cheaper to fix Challenges in Agile Testing……
• 3.
   Volume andSpeed of Requirement Challenges in Agile Testing…… Caused by: Changing requirements, even late in development Changes in definition of product and its correct behavior Resolved by: Risk-based Testing Automated Testing Lightweight reactive Testing Keep test team informed of such changes Account for differential impact of changes on testing
• 4.
   Over-commitment andSprint durations Challenges in Agile Testing…… Caused by: Some projects ritualize the rules, particularly sprint time deadlines Ex: if a project follows four week sprints and continually over-commits, the test team gets squeezed on the last weekend of every sprint Resolved by: Requires team and management maturity Risk-based testing helps the test team deal with over-commitment  Reduce the scope of testing based on risk  Slip low-risk tests into next sprint
• 5.
   Inconsistent andInadequate Unit Testing Challenges in Agile Testing…… Caused by: Unit testing has limited bug-finding effectiveness, averaging 25 to 30%, while good system testing averages around 85% Not all programmers do unit testing Resolved by: Good Unit Testing is the only solution
• 6.
   Broken codedue to frequent builds Caused by: Daily build introduces the rick of breaking code/ existing functionality Resolved by: Perform continuous integration – automated process for creating software build Employ an automation tool that performs regression testing whenever you launch a new build Employ Test-Driven development [TDD] Ensure that Automated Test team is working optimally  Review Automated Test metrics  Use Source Control for reviewing changes to automated tests and protection  Continue to build on your Automated Tests Track metrics are most important for successful automated testing: Metrics for successful automated testing: Feature Coverage Requirement/Feature blocked Daily Test Run trending Daily Test Runs by Host Challenges in Agile Testing……
• 7.
   Inadequate TestCoverage Caused by: Few test cases written for User story Lack of visibility of code check-ins for regular corrections to code Resolved by: Creating test cases at the level of User-story Design sufficient number of tests:  Positive Tests/ Negative Tests/ Performance Tests/ Ancillary Tests Branch code using SCM; any code outside the target always remains untouched Adding triggers to SCM to inform team of code changes Integration with source check-in to find changed code that was not anticipated or planned for Analyzing specific metrics to identify traceability and missing test coverage Metrics for traceability and test coverage: Traceability Coverage Blocked Tests Test Runs by Requirement [User Story] & Configuration Daily Test Run Trending Defects by Requirement/ Daily Defect Trending/ Defect Duration Challenges in Agile Testing……
• 8.
   Inadequate Testingfor published API Caused by: API testing often is omitted because of the misperception that it takes programming skills to call the properties and methods of your API. Resolved by: Dedicated resource High Use functions Usability Tests Security Tests Stopwatch-level performance Tests API metrics to be watched: Function Coverage Blocked Tests Coverage within Function Daily API Test Run Trending Ensure API testing team is working optimally: Review API Run Metrics Continue to build on your API Tests Challenges in Agile Testing……
• 9.
   Poor, changingand missing Test Oracles Challenges in Agile Testing…… Caused by: Agile de-values documentation Change the definition of correct behavior after test execution has started Change the definition of correct behavior without involving the test team Resolved by: Embracing change Conveying information [in] face-to-face conversation Change Management needs to be in-place
• 10.
   Detecting defectsearly Challenges in Agile Testing…… Problems from late-detection of defects:  More time to fix  More money to fix. Resolved by: Performing peer reviews of source code and test artifacts to find early stage defects  Peer Code Review Metrics:  Lines of Code Inspected  Requirement Review Coverage  Manual Test Review Coverage  Automated Test Review Coverage  Code Review Coverage  Static Analysis Issues found  Defects discovered by Peer review Using static analysis tools to identify early stage defects Analyzing peer review statistics and defect aging to address defects early when they’re least costly to fix  Perform Peer reviews  Automatically run static analysis  Review Defect metrics
• 11.
  Challenges in AgileTesting…… Testers and Test managers must help their organizations understand and manage the challenges to manage expectations