Weird collection stats when filtering from pytest_collection_modifyitems hook #12663
Description
setup: platform linux -- Python 3.11.0, pytest-8.3.2
I'm trying to work with the pytest_collection_modifyitems hook, but got in a rabbit hole trying to understand the collected/selected/deselected stats
minimal reproduction example as follows.
conftest.py, which defines a --fancy-k option to filter based on nodeid, much like the existing -k
def pytest_addoption(parser): parser.addoption("--fancy-k") def pytest_collection_modifyitems(config, items): k = config.getoption("--fancy-k") if k: items[:] = [n for n in items if k in n.nodeid]the tests
import pytest @pytest.mark.parametrize("fruit", ["apple", "banana", "coconut", "dragonfruit", "elderberry"]) def test_fruit(fruit): assert fruitTo set a baseline, a default run (I'll omit the non-relevant output lines):
❯ pytest collecting ... collected 5 items test_fruit.py::test_fruit[apple] PASSED [ 20%] test_fruit.py::test_fruit[banana] PASSED [ 40%] test_fruit.py::test_fruit[coconut] PASSED [ 60%] test_fruit.py::test_fruit[dragonfruit] PASSED [ 80%] test_fruit.py::test_fruit[elderberry] PASSED [100%] ============================== 5 passed in 0.00s =============================== A run with classic -k, selecting tests with "n" in their name:
❯ pytest -v -k=n collecting ... collected 5 items / 2 deselected / 3 selected test_fruit.py::test_fruit[banana] PASSED [ 33%] test_fruit.py::test_fruit[coconut] PASSED [ 66%] test_fruit.py::test_fruit[dragonfruit] PASSED [100%] ======================= 3 passed, 2 deselected in 0.00s ======================== Nothing weird so far of course
Using my --fancy-k option now for same query:
❯ pytest -v --fancy-k=n collecting ... collected 5 items test_fruit.py::test_fruit[banana] PASSED [ 33%] test_fruit.py::test_fruit[coconut] PASSED [ 66%] test_fruit.py::test_fruit[dragonfruit] PASSED [100%] ============================== 3 passed in 0.00s =============================== First somewhat off-putting observation: at top it says just "collected 5 items" and footer says "3 passed". Nothing really wrong, but the previous stats with standard -k were more coherent.
Next step: mix --fancy-k with -k (get tests with "n" and "o" in their nodeid):
❯ pytest -v --fancy-k=n -k=o collecting ... collected 5 items / 1 deselected / 4 selected test_fruit.py::test_fruit[coconut] PASSED [ 50%] test_fruit.py::test_fruit[dragonfruit] PASSED [100%] ======================= 2 passed, 1 deselected in 0.00s ======================== Now the collection and summary stats lost coherence:
- "collected 5 items / 1 deselected / 4 selected"
- "2 passed, 1 deselected"