Chapter 2 - Testing in Agile

Testing in Agile 1 Requirement Engineering 2 Testing in Agile 3 Test Automation Software Testing - ISTQB Advance Agile Technical Tester Exam Preparation Chapter 2 Neeraj Kumar Singh 4 Deployment & Delivery

Testing in Agile Contents 2.1 Agile development and test techniques 2.2 Experience-based testing in Agile 2.3 Aspects of code quality Neeraj Kumar Singh

Testing in Agile Contents 2.1.1 Apply test-driven development 2.1.2 Apply behavior-driven development 2.1.3 Analyze a product backlog to introduce acceptance test-driven development Neeraj Kumar Singh

Agile Development and Test Techniques Applying Behavior Driven Development According to the ISTQB® Agile Foundation syllabus, behavior-driven development (BDD) is a technique in which developers, testers, and business representatives work together to analyze the requirements of a software system, formulate them using a shared language, and verify them automatically. BDD is strongly influenced by two separate disciplines: test-driven development (TDD) and domain driven design (DDD). It incorporates many of the core principles of both disciplines, including collaboration, design, ubiquitous language, automated test execution, short feedback cycles, and more. TDD relies on unit tests to verify implementation details whereas BDD relies on executable scenarios to verify behaviors and more. BDD generally follows a prescribed path:  Create user stories collaboratively  Formulate user stories as executable scenarios and verifiable behaviors  Implement behaviors and execute the scenarios to verify them Teams that apply BDD extract one or more scenario from each user story and then formulate them as automated tests. A scenario represents a single behavior under specific conditions. Scenarios are typically based on user story’s acceptance criteria, examples, and use cases. BDD scenarios should be written using language that can be understood by all team members, whether technical or non-technical. Neeraj Kumar Singh

Agile Development and Test Techniques Applying Behavior Driven Development BDD scenarios are typically composed of three main sections : 1. Given – describes the state of the environment (preconditions) before the behavior is triggered 2. When – describes the actions that trigger the behavior 3. Then – describes the expected results of the behavior Extracting scenarios from user stories involves:  Identifying all acceptance criteria specified by a user story and writing scenarios for each criterion.  Identifying functional use cases and examples and writing scenarios for each of them.  Creating scenarios while testing exploratory, which can help identifying related existing behaviors, potential conflicting behaviors, minimal states, alternative flows, etc.  Looking for repeated use of steps or groups of steps to avoid repetitive work.  Identifying areas that require random or synthetic data.  Identifying steps that require mocks, stubs or drivers to maintain isolation and to avoid executing integrations or possibly costly processes.  Ensuring scenarios are atomic and do not affect each other’s state (isolated).  Deciding whether to limit the When sections to one step in accordance with the principle that every test checks just one thing, or to optimize for other considerations, such as test execution speed. Neeraj Kumar Singh

Agile Development and Test Techniques Applying Behavior Driven Development Recommended guidelines for formulating scenarios include:  The scenario should describe a specific behavior that the system supports from the perspective of a specific user.  The scenario should use the third person when describing the steps (Given, When, Then) to describe the state and interactions from the perspective of the user.  The scenarios should be isolated and atomic so that they can be run in any order and not affect each other or rely on each other.  The When steps should describe the semantic actions that a user performs rather than the specific technical actions, unless there is a particular need to test a specific action. For example, “The user confirms the order” (a semantic action) is generally preferable to “The user clicks the Confirm button” (a technical action) unless the button itself has to be tested.  The Then steps should describe specific observations or states. Neeraj Kumar Singh

Agile Development and Test Techniques Applying Acceptance Test Driven Development Acceptance test-driven development (ATDD) supports the coordination of software projects to address the delivery based on the customer needs. Acceptance tests are specifications for the desired behavior and functionality of a system. A user story represents a piece of customer-valued functionality. Acceptance tests verify that this functionality is implemented correctly. This user story is split by the developers into a set of tasks required to implement that functionality. Developers can implement these tasks using TDD. When a given task is completed, the developers move on to the next task, and so forth, until the user story is completed, which is indicated by successfully executed acceptance tests. Both BDD and ATDD are customer focused whereas TDD is developer focused. BDD and ATDD are similar in that they both produce the same result: a shared understanding of what is to be built and how to build the thing right. BDD is a structured way of writing test cases Neeraj Kumar Singh

Agile Development and Test Techniques Applying Acceptance Test Driven Development ATDD separates the test intention, which is expressed in a human readable format such as plain text, from the test implementation, which is preferably automated. This important separation means that business-facing and other non-technical team members such as product owners, analysts, and testers can take an active role in describing testable examples (or acceptance tests) to drive development. Stakeholders with different roles and perspectives, such as customer, developer, and tester come together to collaboratively discuss a potential user story (or other form of requirement) to reach a shared understanding of the problem to be solved, to ask each other open questions regarding the functionality and to explore concrete, testable examples of required behavior. The agreed examples are valuable outputs in themselves, so it is important to remember that although they may also be automated as acceptance tests, it is not always necessary or cost-effective to do so. Neeraj Kumar Singh

Agile Development and Test Techniques Applying Acceptance Test Driven Development This illustrates an important role for the Agile tester in this situation. During the discussions, the tester may be thinking in terms of test techniques, such as equivalence partitioning, boundary value analysis, and so on. They can use this analytical approach to encourage questions to be asked to the product owner about where the interesting behaviors and edge cases can be found. Specification by example (or SBE) refers to a collection of useful patterns that allow an Agile team to discover, discuss, and confirm the important outcomes required by business stakeholders and the software behaviors that are needed to reach those outcomes. The term has been widely used to include and broaden the meaning of the term acceptance test-driven development (ATDD). Neeraj Kumar Singh

Experience Based Techniques in Agile Contents 2.2.1 Combining experience-based techniques and black-box tests 2.2.2 Creating test charters and interpreting their results Neeraj Kumar Singh

Experience Based Techniques in Agile Combining Experience based and Black box Techniques The various characteristics of Agile projects such as the approach, length of iteration, applicable test level(s), risk level of project and product, quality of requirements, level of experience/expertise of the team members, project organization, etc. will influence the balance of automated tests, exploratory tests, and manual black-box tests in an Agile project. During risk analysis, the risk levels (e.g. high, medium, low) are determined for the individual system features and functionality. The next step is to find the right mix and balance of automated tests, exploratory tests, and manual black-box tests for a specific risk level. Neeraj Kumar Singh

Experience Based Techniques in Agile Combining Experience based and Black box Techniques Below is a table describing this idea. In this table, the risk levels are listed vertically and the three test approaches horizontally. The table below is an example of a mix of different testing techniques (exploratory, automated, and manual) that can be used in a safety critical system. This table can be adapted to other specific projects. ++ (highly recommended) + (recommended) o (neutral) - (not recommended) -- (not to be used) When looking at the first row, it suggests that in this situation, a combination of automated test and black-box test would be highly recommended in addition to an exploratory testing approach. The decision to automate (or not) will also be influenced by many other factors. Neeraj Kumar Singh Safety Critical System

Experience Based Techniques in Agile Combining Experience based and Black box Techniques The following is an example of a combination of different testing techniques when used in a non-safety critical system. ++ (highly recommended) + (recommended) o (neutral) - (not recommended) -- (not to be used) When looking at the last row, it suggests that in this situation, an exploratory testing approach is highly recommended, while other approaches might not be used. In any situation (safety critical or not) the specific mixture will depend on the project characteristics Neeraj Kumar Singh Non-Safety Critical System

Experience Based Techniques in Agile Creating Test Charters and Interpreting Their Results Before an appropriate test charter can be created, the existing epics and user stories should be evaluated first. When analyzing epics or user stories to create a test charter the following things should be considered:  Who are the users in this epic or user story?  What is the main functionality of the epic or user story?  What are the actions a user can perform? (This can be obtained from the acceptance criteria list, which is defined for a user story)  Is the goal of the user story realized once the feature or functionality is complete? (Or are there other testing tasks affecting the Definition of Done) The granularity of a test charter is important. It should not be too small, as it must explore an area around an identified problem (re-active, regression) or an area around a user story or an epic (proactive, revealing). It shouldn’t be too big, as it should fit within a 60 to 120 minutes time box. The goal of running an exploratory testing session is to help us make a good quality decision, regarding a phenomenon, bug clustered area, etc.. Neeraj Kumar Singh

Experience Based Techniques in Agile Creating Test Charters and Interpreting Their Results  Test charters can be created using flipcharts, spreadsheets, documents, existing test management system, personas, mind mapping and a whole team approach. Exploratory testers use heuristics to drive their creativity in writing and performing exploratory testing sessions. These can also be used for creating test charters and in thinking creatively when analyzing user stories and epics.  All the findings found during exploratory testing should be documented. The results of exploratory testing should provide insights into better test design, ideas for testing the product, and ideas for any further improvement. Findings that should be documented during exploratory testing include defects, ideas, questions, improvement suggestions, etc.  Tools can be used for documenting the exploratory testing sessions. These include video capture and logging tools, planning tools, etc. The documentation should include the expected result. In some cases, pen and paper is sufficient, depending on the volume of information to collect.  When summarizing the exploratory testing session, during the debriefing meeting the information is collected and aggregated to present a status of progress, coverage, and efficiency of the session. This summary information can be used as management report, or used in retrospective meetings on any level and any scale.  However, it can be quite challenging to determine accurate test metrics related to exploratory testing sessions. Neeraj Kumar Singh

Aspects of Code Quality Contents 2.3.1 Refactoring 2.3.2 Code reviews and static code analysis to identify defects and technical debt Neeraj Kumar Singh

Aspects of Code Quality Refactoring Refactoring is a way to clean up code in an efficient and controlled manner, by clarifying and simplifying the design of existing code and test cases, without changing its behavior. In Agile projects, the iterations are short, which creates a short feedback loop for all team members. Short iterations also create a challenge for testers trying to achieve adequate coverage. Due to the nature of the iterations, the fact that functionality is growing, and features are added and enhanced over time, test cases that have been written for a feature in an earlier iteration, often need maintenance or even complete redesign in later iterations. After user stories are understood and acceptance criteria are written for each of them, the impact of the functionality of the current iteration on the existing regression tests (manual and automated) can be analyzed, and refactoring and/or enhancing of the tests may be required. Teams are maintaining and extending their code extensively from iteration to iteration, and without continuous refactoring, this is hard to do. Neeraj Kumar Singh

Aspects of Code Quality Refactoring The refactoring of the test cases can be done as follows:  Identification: Identifying existing tests that require refactoring by reviewing or causal analysis.  Analysis: Analyzing the impact of the changed tests on the overall regression test set.  Refactor: Making changes to the internal structure of the tests to make it easier to understand and cheaper to modify without changing its observable behavior.  Re-run: Re-running the tests, checking their results and documenting defects where relevant. The refactoring should not have influenced the result of test execution.  Evaluate: Checking the results of the re-run tests, ending this phase when the tests have passed a quality threshold defined and accepted by the team. Neeraj Kumar Singh

Aspects of Code Quality Code Review and Static Code Analysis A code review is a systematic examination of code by two or more people (one of whom is usually the author). Static code analysis is the systematic examination of code by a tool. Both are effective and widespread practices for exposing and identifying issues that affect code quality. Code reviews and static code analysis provide constructive feedback that helps identify defects and manage technical debt. Impediments such as resource constraints, higher technical complexity than expected, rapidly changing priorities, and technical limitations may hinder efforts to write quality code and force programmers to make compromises that will decrease the quality of the code in favour of more immediate results. These compromises can introduce defects and incur technical debt. Technical debt refers to the increased effort that will be required to implement a better solution (including removing latent defects) in the future as a consequence of choosing an inferior but easierto-implement solution now. Technical debt is often incurred unintentionally, by subtle compromises or a gradual accumulation of small or unnoticed changes as the software evolves. Besides helping identify defects and manage technical debt, code analysis and reviews offer additional benefits:  Training and sharing knowledge  Improving the robustness, maintainability, and readability of the code  Providing oversight and maintaining consistent coding standards Neeraj Kumar Singh

Aspects of Code Quality Code Review and Static Code Analysis Code Review By participating in code reviews, testers can use their unique perspective to make valuable contributions to code quality by collaboratively working with programmers to identify potential defects and avoid technical debt at a very early stage. They can bring their expertise to bear in many ways, such as by asking questions about the behavior of the code, suggesting use cases that may not have been considered, or monitoring code metrics that could indicate quality issues. Code reviews are manual activities, possibly supported by tools, that are performed by or with other people (in addition to the author). Code reviews may be performed with the reviewer and the author/developer sitting side-by-side. This mode, which is common in ad-hoc reviews and pair programming, facilitates excellent communication and encourages deeper analyses and better knowledge sharing. On distributed teams or teams that prefer a more disconnected approach, the code review process is facilitated by the configuration management system. The process is usually partially automated as part of the continuous integration process. These processes may support code reviews with a single reviewer in each round of review, or collaborative team reviews. Neeraj Kumar Singh

Aspects of Code Quality Code Review and Static Code Analysis Static Code Analysis In static code analysis, a tool analyzes the code and searches for specific issues without executing the code. The results of static code analysis may point at clear issues in the code or provide indirect indicators that require further assessment. Many development tools, especially integrated development environments (IDEs), can perform static code analysis while writing the code. This provides the benefit of immediate feedback, though it may only be possible to apply just a subset of the analysis performed during continuous integration. Neeraj Kumar Singh

Testing in Agile Sample Questions Pattern Neeraj Kumar Singh Source : www.istqb.org

Testing in Agile Sample Questions 1. Summarize the characteristics of performing exploratory tests with test charters. Select ONE options. Answer Set a. In contradiction to black-box testing, the expected result is documented after a defect is found and not as part of the test design. b. Test charters are a useful tool to be used for testing, when a detailed specification to the system under test is given. c. The result of performing exploratory testing by using test charters is finding defects and specification error. d. Exploratory testing and black-box testing use the same metrics, when it comes to measure the test coverage. Neeraj Kumar Singh

Testing in Agile Sample Questions 2. You do have to analyze the following Bubble Sort function written in Java: Analyzation should be done according to the following checklist: 1. All Variables should start with a Capital letter, except they are used only as counting variable 2. If the name of the variable consists of more than one word, all words have to start with Capital letters (e.g. FirstName) 3. All constants should be written in CAPITAL letters, except they are only used as start or end point (e.g. in loops) Which one of the above mentioned checkpoint items is not fulfilled? . Select ONE options. Answer Set a. 2 b. 1 c. 3 d. None Neeraj Kumar Singh

Testing in Agile Sample Questions 3. You, as a tester in a BDD environment, know that the following user story has to be implemented and tested: As a Scrum Master I want to see Lead/Cycle time progress So that I know whether we are improving our development process or not. Which one of the following BDD scenarios in Gherkin format is at least partially, but best fitting to this user story? Select ONE options. Answer Set a. Given Reports section in Project area and Bug Tracking practice is DISABLED When I navigate to Lead and Cycle Time Report Then I see Lead Time chart And chart contains 1 line for stories. b. Given Reports section in Project area and Bug Tracking practice is ENABLED If I navigate to Lead and Cycle Time Report Then I see Cycle Time And chart contains 2 lines (for stories and bugs) Else Error message is popping up. c. Given Reports section in Project area and Bug Tracking practice is DISABLED When I navigate to Lead and Cycle Time Report Then I see the chart And chart contains information for all stories. d. When Reports section in Project area and Bug Tracking practice is ENABLED Then I navigate to Report section And I see Cycle Time with chart containing 2 lines (for stories and bugs). Neeraj Kumar Singh

Chapter 2 - Testing in Agile

More Related Content

What's hot

Similar to Chapter 2 - Testing in Agile

More from Neeraj Kumar Singh

Recently uploaded

Chapter 2 - Testing in Agile