pytest plugin: Improve performance via caching #472
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
tony force-pushed the pytest-plugin-fixture-caching branch 4 times, most recently from a1cea48 to 4e66283 Compare 
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@ ## master #472 +/- ## ========================================== + Coverage 63.85% 64.17% +0.32% ========================================== Files 40 40 Lines 3591 3713 +122 Branches 774 788 +14 ========================================== + Hits 2293 2383 +90 - Misses 772 793 +21 - Partials 526 537 +11 ☔ View full report in Codecov by Sentry. |
tony force-pushed the pytest-plugin-fixture-caching branch 3 times, most recently from e6bcfba to ae8e219 Compare tony force-pushed the pytest-plugin-fixture-caching branch 4 times, most recently from 68a8bd8 to 3b4d3c0 Compare tony force-pushed the pytest-plugin-fixture-caching branch from 3b4d3c0 to b7c4149 Compare tony force-pushed the pytest-plugin-fixture-caching branch from b7c4149 to faaba05 Compare tony force-pushed the pytest-plugin-fixture-caching branch from 7af8ea6 to a658c43 Compare This was referenced Oct 12, 2024
tony added a commit that referenced this pull request Oct 12, 2024
_Resolves #474_ # Problem Issue #472 requires session-scoped fixtures. Currently, we use `set_home`, `gitconfig`, and `hgconfig` to `monkeypatch.setenv` `$HOME`, pointing it to a temporary directory with user-specific configuration files. This ensures all subsequent `git` and `hg` commands automatically load these configurations. However, [`monkeypatch.setenv`](https://docs.pytest.org/en/8.3.x/reference/reference.html#pytest.MonkeyPatch.setenv) doesn't work with function-scoped fixtures. # Improvement ``` ❯ hyperfine \ --warmup 3 \ --runs 10 \ --prepare 'git checkout master' \ --command-name 'libvcs 0.31.0' \ 'py.test' \ --prepare 'git checkout pytest-config' \ --command-name 'with improved hg/git config fixtures' \ 'py.test' Benchmark 1: libvcs 0.31.0 Time (mean ± σ): 15.150 s ± 0.751 s [User: 16.650 s, System: 4.720 s] Range (min … max): 14.235 s … 16.741 s 10 runs Benchmark 2: with improved hg/git config fixtures Time (mean ± σ): 15.014 s ± 0.307 s [User: 17.246 s, System: 4.865 s] Range (min … max): 14.458 s … 15.642 s 10 runs Summary with improved hg/git config fixtures ran 1.01 ± 0.05 times faster than libvcs 0.31.0 ``` # Changes ## pytest fixtures: Session-scoped `hgconfig` and `gitconfig` These are now set by `set_hgconfig` and `set_gitconfig`, which set `HGRCPATH` and `GIT_CONFIG`, instead of overriding `HOME`.
tony added a commit to vcs-python/vcspull that referenced this pull request Oct 13, 2024
# Problem Git, Subversion, and Mercurial repositories are unnecessarily reinitialized for each test via libvcs' pytest plugin. - We're not utilizing session-based scoping. - A single initial repo could be created, then copied to [`tmp_path`](https://docs.pytest.org/en/8.3.x/how-to/tmp_path.html#the-tmp-path-fixture) using [`shutil.copytree`](https://docs.python.org/3/library/shutil.html#shutil.copytree) ([source](https://github.com/python/cpython/blob/v3.13.0/Lib/shutil.py#L550-L605)). # Improvement ``` ❯ hyperfine \ --warmup 3 \ --runs 10 \ --prepare 'pip uninstall -y libvcs && pip install "libvcs==0.30.1"' \ --command-name 'libvcs 0.30.1' \ 'py.test' \ --prepare 'pip uninstall -y libvcs && pip install "libvcs==0.31.0"' \ --command-name 'libvcs 0.31.0' \ 'py.test' \ --prepare 'pip uninstall -y libvcs && pip install "libvcs==0.32.1"' \ --command-name 'libvcs 0.32.1' \ 'py.test' Benchmark 1: libvcs 0.30.1 Time (mean ± σ): 4.784 s ± 0.116 s [User: 4.641 s, System: 0.846 s] Range (min … max): 4.599 s … 4.971 s 10 runs Benchmark 2: libvcs 0.31.0 Time (mean ± σ): 3.436 s ± 0.115 s [User: 2.821 s, System: 0.470 s] Range (min … max): 3.347 s … 3.678 s 10 runs Benchmark 3: libvcs 0.32.1 Time (mean ± σ): 3.173 s ± 0.054 s [User: 2.726 s, System: 0.388 s] Range (min … max): 3.104 s … 3.295 s 10 runs Summary libvcs 0.32.1 ran 1.08 ± 0.04 times faster than libvcs 0.31.0 1.51 ± 0.04 times faster than libvcs 0.30.1 ``` # Changes ## libvcs 0.30.1 -> 0.32.1 See also: - https://github.com/vcs-python/libvcs/blob/v0.32.1/CHANGES#libvcs-0310-2024-10-12 - https://libvcs.git-pull.com/history.html#libvcs-0-32-1-2024-10-12 # Appendix - vcs-python/libvcs#472
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Problem
Git, Mercurial, and Subversion repositories are unnecessarily reinitialized for each test.
tmp_pathusingshutil.copytree(source).Issue #471 highlighted this inefficiency, where benchmarks showed tens of thousands of redundant function calls.
Improvement
Changes
Pytest fixtures overhaul