Report for Assignment 1
Project chosen
Name: Python-Prompt-Toolkit
URL: https://github.com/prompt-toolkit/python-prompt-toolkit
Number of lines of code and the tool used to count it: 39446 lines. Tool used: Lizard Programming language: Python
Coverage measurement
Existing tool
We used the existing tool called Coverage.py to measure the coverage. To do so we
Installed the following Python packages: coverage, pytest. To measure the coverage in the source directory, we first needed to run the tests, using the command “coverage run -m pytest”. To see the results and coverage, we executed “coverage report” to see it in the command line, and ‘coverage html --omit="tests/" --include="src/"’ to see a more detailed HTML version of the coverage.
Kirill Nikolaevskii Own coverage tool:
“_commonprefix” from “src/prompt_toolkit/completion/base.py”
Screenshot of the coverage results output by the instrumentation:
“is_dimension” from “src/prompt_toolkit/layout/dimension.py”
Link:
Screenshot of the coverage results output by the instrumentation:
Coverage improvement: Test 1.
Screenshot of the old coverage results:
Screenshot of the new coverage results:
Overview:
The coverage was improved by 100% because the initial set of tests did not hit any of the conditional branches of the _commonprefix function. The enhanced tests used additional cases to call the tested function with inputs that hit all the branches. The input of an empty array of strings hits the first branch, which is triggered if there are no strings. The input of two equal strings ['car', 'car'] led to hitting the second branch, which is triggered by a non-empty array of strings.
Finally, the third branch was hit with the input ['car', 'dog'], which passed through the else statement for the second branch and then led to the execution of the third branch, as it expects that some character of the minimal string is different from the maximum string at the same index. Thus, the coverage was increased by 100% for this function as the new cases hit all branches.
Test 2.
Screenshot of the old coverage results:
Screenshot of the new coverage results:
Overview:
The coverage was improved by 100% because the initial set of tests did not hit any of the conditional branches of the is_dimension function. The enhanced test executed additional cases that led to hitting all conditional branches. For instance, the input of a None value triggered the execution of Branch 1, which is an if statement that expects such input. Additionally, the integer input led to the execution of Branch 3, which has an if statement expecting an integer.
Moreover, the input of a regular function led to the execution of Branch 2, which has an if statement for a callable function. Lastly, when there was a string input, Branch 4 was executed as it is the default case when the input of the is_dimension function is incorrect. In conclusion, the coverage was improved by adding 4 use cases that hit all the previously unexecuted conditional branches.
Rovshan Tamirov Own coverage tool:
“to_dimension” from “src/prompt_toolkit/layout/dimension.py”
Screenshot of the coverage results output by the instrumentation:
“merge_processors” from “src/prompt_toolkit/layout/processors.py”
Screenshot of the coverage results output by the instrumentation:
Coverage improvement: Test 1.
Screenshot of the old coverage results:
Screenshot of the new coverage results:
Overview:
The reason for the improvement is as follows. Mainly, since the original repository didn’t test out the function mentioned above, the coverage without additional tests was 0%. After adding the additional tests focused on each of the branches, the coverage increased by 100%.
Test 2.
Screenshot of the old coverage results:
Screenshot of the new coverage results:
Overview:
The reason for such an improvement in the coverage has already been mentioned above. The original repository didn’t test out the function mentioned above, the coverage without additional tests was 0%. After adding the additional tests focused on each of the branches, the coverage increased by 100%.
Daniil Malygin Own coverage tool:
“find_next_matching_line” from “document.py”
Link: https://github.com/knikolaevskii/SEP_python-prompt-toolkit/commit/ae26bfdf11366217cac1
Screenshot of the coverage results output by the instrumentation:
“find_previous_matching_line” from “document.py”
Screenshot of the coverage results output by the instrumentation:
Coverage improvement: Test 1.
Screenshot of the old coverage results:
Screenshot of the new coverage results:
Overview:
The coverage was improved by 100% because the original set of tests was empty and did not cover any of the conditional branches of the find_next_matching_line function. The enhanced test suite executed additional cases that led to hitting all conditional branches. For example, the input text with multiple empty lines triggered the execution of Branch 1, which is an if statement that expects such input to find the second next empty line.
In conclusion, the coverage was improved by adding 3 use cases that hit all the previously unexecuted conditional branches, ensuring comprehensive testing of the find_next_matching_line function.
Test 2.
Screenshot of the old coverage results:
Screenshot of the new coverage results:
Overview:
The coverage was improved by 100% because the original set of tests was empty and did not cover any of the conditional branches of the find_prev_matching_line function. The enhanced test suite executed additional cases that led to hitting all conditional branches. For example, the input text with multiple empty lines triggered the execution of Branch 1, which is an if statement that expects such input to find the second previous empty line.
In conclusion, the coverage was improved by adding 3 use cases that hit all the previously unexecuted conditional branches, ensuring comprehensive testing of the find_previous_matching_line function.
Bence Matajsz Own coverage tool:
Insert_line_above from buffer.py
Link: https://github.com/prompt-toolkit/python-prompt-toolkit/commit/daeccc3cb0855bfd87a9a3e4c1cc4cb bf965c661
Screenshot of the coverage results output by the instrumentation:
translate_row_col_to_index from document.py
Link: https://github.com/prompt-toolkit/python-prompt-toolkit/commit/daeccc3cb0855bfd87a9a3e4c1cc4cb bf965c661
Screenshot of the coverage results output by the instrumentation:
Coverage improvement: Test 1.
Link: https://github.com/prompt-toolkit/python-prompt-toolkit/commit/daeccc3cb0855bfd87a9a3e4c1cc4cb bf965c661
Screenshot of the old coverage results:
Screenshot of the new coverage results:
Overview:
The default coverage was 0%, as there were no tests checking branch coverage for this function. After adding test cases covering each branch, I managed to get the coverage up to 100%. There are two possibilities, the copy_margin argument of the function either being true or false. Both cases are tested, ensuring the function is covered in either case.
Test 2.
Link: https://github.com/prompt-toolkit/python-prompt-toolkit/commit/daeccc3cb0855bfd87a9a3e4c1cc4cb bf965c661
Screenshot of the old coverage results:
Screenshot of the new coverage results:
Overview:
The default coverage was 0%, as there were no tests checking branch coverage for this function. After adding test cases covering each branch, I managed to get the coverage up to 100%. There are multiple branches in this function, so I created 9 test cases to make sure everything is covered well. These test cases ensure that all branches are covered, including all try-except and if-else blocks. The remaining code after the try-except block is also checked, making sure it is executing properly. A wide variety of arguments are being checked, including out-of-range values.
Screenshot of the old coverage results by running an existing tool:
Screenshot of the new coverage results by running the existing tool using all test modifications made by the group:
Kirill Nikolaevskii
- Own Coverage Tool: Improved coverage for the _commonprefix function in src/prompt_toolkit/completion/base.py and the is_dimension function in src/prompt_toolkit/layout/dimension.py by writing additional tests and documenting the results.
Rovshan Tamirov
- Coverage Improvement: Focused on the to_dimension function in src/prompt_toolkit/layout/dimension.py and the merge_processors function in src/prompt_toolkit/layout/processors.py. Added tests to improve coverage, documented the results, and provided screenshots showing the coverage improvement.
Daniil Malygin
- Coverage Improvement: Worked on the find_next_matching_line and find_previous_matching_line functions in document.py. Added tests to ensure comprehensive branch coverage and documented the coverage improvements with screenshots.
Bence Matajsz
- Coverage Improvement: Improved coverage for the insert_line_above function in buffer.py and the translate_row_col_to_index function in document.py. Added tests to cover all branches, including various cases to ensure thorough testing. Documented the results and provided screenshots of the coverage improvements.