2-Way Coverage vs 3-Way Coverage
- 1.
- 2. Each of theend points on this mind map (like “Young” and “Friendly”) are parameter values. Parameters Parameter Values
- 3. This is whatthey look like in Hexawise. Parameters Parameter Values
- 4. When you clickon the “Create Tests” button... 1 Hexawise generates a set of 2-way tests that contains every single pair of those test parameter values in at least one of the tests. 2
- 5. • Medical Devices: D.R. Wallace, D.R. Kuhn, Failure Modes in Medical Device Software: an Analysis of 15 Years of Recall Data, International Journal of Reliability, Quality, and Safety Engineering, Vol. 8, No. 4, 2001. • Browser, Server: D.R. Kuhn, M.J. Reilly, An Investigation of the Applicability of Design of Experiments to Software Testing, 27th NASA/IEEE Software Engineering Workshop, NASA Goddard SFC 4-6 December, 2002 . • NASA database: D.R. Kuhn, D.R. Wallace, A.J. Gallo, Jr., Software Fault Interactions and Implications for Software Testing, IEEE Trans. on Software Engineering, vol. 30, no. 6, June, 2004. • Network Security: K.Z. Bell, Optimizing Effectiveness and Efficiency of Software Testing: a Hybrid Approach, PhD Dissertation, North Carolina State University, 2006. 1 input or 2 inputs (“pairwise) 3 inputs 4, 5 or 6 inputs Studies show that 84% of defects in production could have been found by testing for every pair of parameter values. 2-way coverage is useful because so many defects can be quickly found by testing all pairs of parameter values.
- 6. But what does2-way coverage mean EXACTLY?
- 7. It means this... Everysingle pair of parameter values appears together in at least one of these six test cases:
- 8.
- 9. Stylish and Tax Collector? Test #2 That’s a pair of values. So at least one test includes it.
- 10.
- 11. Old and Shabby? Test # 4 That’sa pair of values. So at least one test includes it.
- 12.
- 13. Hostile and Skydiving Instructor? Also test #4 That’s a pair of values. So at least one test includes it.
- 14.
- 15. Young and Hostile? Test # 6 That’sa pair of values. So at least one test includes it.
- 16. We could dothis for quite a while. Choose any two parameter values. Any pair of test inputs you choose will appear in at least one of the 2-way tests.
- 17. For every setof Parameter Values entered into Hexawise there is a finite set of pairs that can be tested. There are exactly 24 possible pairs of parameter values in this simple example.
- 18. All of thepairs will be tested in at least one test. All 24 of those possible pairs are tested in at least one of the following 6 tests.
- 19. • Medical Devices: D.R. Wallace, D.R. Kuhn, Failure Modes in Medical Device Software: an Analysis of 15 Years of Recall Data, International Journal of Reliability, Quality, and Safety Engineering, Vol. 8, No. 4, 2001. • Browser, Server: D.R. Kuhn, M.J. Reilly, An Investigation of the Applicability of Design of Experiments to Software Testing, 27th NASA/IEEE Software Engineering Workshop, NASA Goddard SFC 4-6 December, 2002 . • NASA database: D.R. Kuhn, D.R. Wallace, A.J. Gallo, Jr., Software Fault Interactions and Implications for Software Testing, IEEE Trans. on Software Engineering, vol. 30, no. 6, June, 2004. • Network Security: K.Z. Bell, Optimizing Effectiveness and Efficiency of Software Testing: a Hybrid Approach, PhD Dissertation, North Carolina State University, 2006. 1 input or 2 inputs (“pairwise) 3 inputs 4, 5 or 6 inputs It is an extremely efficient way to find defects given that: •Studies show that 84% of defects in production could have been found by testing for every pair of parameter values and •2-way coverage can be achieved in a relatively small number of tests. Again, the reason it is a good idea to test all possible pairs is because...
- 20. But what aboutcoverage of combinations involving 3 or more parameter values?
- 21. It depends! Sometimes, aspecific combination of 3 values will be included in your set of 2- way tests. Other times, combinations of 3 values will not be included in your set of tests. Let’s look at a few examples.
- 22. 3-way combinations involve3 specific parameter values. Young and Stylish and Skydiving Instructor?
- 23. 3-way combinations involve3 specific parameter values. Young and Stylish and Skydiving Instructor? Old and Stylish and Skydiving Instructor? Shabby and Hostile and Tax Collector? Young and Stylish and Hostile? Let’s look at a few examples of 3-way combinations and see if they are covered in our 2-way tests.
- 24. Young and Stylish and Skydiving Instructor? This 3-way combination does appear intest # 3 We got lucky. This 3-way combination does appear in the 2-way tests.
- 25. Old and Stylish and Skydiving Instructor? This 3-way combinationdoes not appear in the 2-way tests.
- 26. Shabby and Hostile and Tax Collector? This 3-waycombination does not appear in the 2-way tests.
- 27. Young and Stylish and Hostile? This 3-way combinationdoes not appear in the 2-way tests.
- 28. If you wantto make sure that all possible “triples” like these are covered by at least one of your tests, you will need to create 3-way tests. Let’s now take a look at the 3-way tests from Hexawise that includes at least one test for every possible combinations of “triples.”
- 29. These 3-way testscontain at least one test for every triple. Some 3-way combinations are covered in 2-way tests.
- 30. All 3-way combinationsare covered in 3-way tests. Young and Stylish and Skydiving Instructor? Old and Stylish and Skydiving Instructor? Shabby and Hostile and Tax Collector? Young and Stylish and Hostile? Test # 6 Test # 3 Test # 8 Test # 4 Reviewing the same 3-way combinations as before, we see that all of them are covered in the 3-way tests. That’s because every single triple is included in the 3- way tests.
- 31. For additional information,please see: training.hexawise.com or help.hexawise.com