This repo is an experiment to demonstrate the different results for code coverage metrics based on different coding styles. The control of the experiment is the main branch.
This repo is in Node/Javascript, and uses Jest for testing. Before running these tests you should have Node.js, preferably the latest, and run the following command;
npm install
Jest by default uses Istanbul for measuring code coverage. To view the results on your local machine as well as properly understand the purpose of these tests, you should do as follows;
npm test -- --coverage
The coverage modifier shows the metrics
This repo contains 6 different versions of a function that does the same thing - it accepts 3 boolean parameters; X, Y & Z. If any of the parameters are true, then each function returns true. Otherwise it returns false.
The main branch has 100% code coverage across all test cases and all types of code coverage including;
- Line coverage
- Statement Coverage
- Branch Coverage
- Function/Method Coverage
you can read more about what each type of coverage is here;
https://www.sonarsource.com/learn/code-coverage/#line-coverage
The functional test cases are as follows;
- X is true
- Y is true
- Z is true
- All parameters are false
The branches for this repo are designed to demonstrate what happens to code coverage metrics when the code remains the same but certain selected tests are disabled. The test scenarios are as follows;
- All tests cases are executed
- Disable test for all parameters are false
- Disable tests for Y and Z are true
- Disable all tests except X is true
- Disable all tests except Z is true
When you change branches and execute the tests you can see how each file with a different style of coding affects the accuracy of code coverage metrics. The more concise coding styles code coverate metrics are more likely to give false impression of good code coverage, as they are closer to 100% even with test cases missing.
The following list are the branches and their corresponding test scenario;
- experiment_01_all_tests_active - All tests cases are executed
- experiment_02_disable_all_false - Disable test for all parameters are false
- experiment_03_disable_y_and_z_are_true - Disable tests for Y and Z are true
- experiment_04_all_tests_except_x_is_true - Disable all tests except X is true
- experiment_05_disable_all_tests_except_z_is_true - Disable all tests except Z is true
The more verbose/explicit the code is however, the more informative and accurate the code coverage metrics are, and the harder it is to fool the coverage tool.
| File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
|---|---|---|---|---|---|
| All files | 100 | 100 | 100 | 100 | |
| condition_control.js | 100 | 100 | 100 | 100 | |
| condition_if_else_grouped.js | 100 | 100 | 100 | 100 | |
| condition_if_else_separate.js | 100 | 100 | 100 | 100 | |
| condition_if_else_then_return_separate.js | 100 | 100 | 100 | 100 | |
| condition_switch_grouped.js | 100 | 100 | 100 | 100 | |
| condition_switch_separate.js | 100 | 100 | 100 | 100 |
Here are the code coverage stats forthe variation with the most compelling results, branch experiment_05_disable_all_tests_except_z_is_true
| File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
|---|---|---|---|---|---|
| All files | 66.66 | 53.57 | 100 | 66.66 | |
| condition_control.js | 100 | 100 | 100 | 100 | |
| condition_if_else_grouped.js | 75 | 80 | 100 | 75 | 5 |
| condition_if_else_separate.js | 62.5 | 50 | 100 | 62.5 | 3,5,9 |
| condition_if_else_then_return_separate.js | 66.66 | 50 | 100 | 66.66 | 4,6,10 |
| condition_switch_grouped.js | 75 | 25 | 100 | 75 | 8 |
| condition_switch_separate.js | 50 | 25 | 100 | 50 | 4-6,10 |
Take note of how the more concise code, in the condition_control.js file, has 100% code coverage despite 3 out of 4 tests having been disabled. That would indicate that the more concise your code is, the easier it is to get an inflated code coverage score with sub standard test coverage.
On the other hand all the other coding style variations do a better job indicating the code coverage requires additional tests.