-
- Notifications
You must be signed in to change notification settings - Fork 2.9k
Description
note that there were major edits of this comment
pytest supports passing arguments to fixture functions using so-called indirect parametrization:
@pytest.fixture def service(request): return f"Service launched with parameter {request.param!r}" @pytest.mark.parametrize("service", ["parameter"], indirect=True) def test_with_service(service): assert service == "Service launched with parameter 'parameter'"There are some problems with this:
- This is buried in the documentation and hard to find
- This makes passing arguments a parametrization api. Parametrization, as I understand, usually refers to running the same test multiple times with different arguments. In the above example
test_with_serviceis only run once. This test may only work with a service launched in this particular way. It may not make sense to run it against different kinds of service. In other words, this test doesn't require parametrization. - The syntax, especially the
indirect=...bit, is rather confusing - The fixture has to manually deal with
request.paramobject. It somehow has to verify that it contains the things that it can use, choose default values if not, and throw exceptions in case of errors. some of this behavior looks very much like regular function calls and could benefit from better syntax
So please add a more obvious way to pass parameters to fixture functions.
Additionally, we should note that there's a similar syntax to parametrize fixtures:
@pytest.fixture(scope="module", params=["smtp.gmail.com", "mail.python.org"]) def smtp_connection(request): smtp_connection = smtplib.SMTP(request.param, 587, timeout=5) ...It is also problematic:
- This is more or less the same idea as with test parametrization, but it uses a different syntax
- Due to the fact that conceptually there is only a single argument (
requst.param), you can't easily create a “matrix” of arguments like you can do withpytest.mark.parametrize. Consider this:This will run the test four times using all possible combinations of arguments. This is not possible with@pytest.mark.parametrize("one", ["1", "uno"]) @pytest.mark.parametrize("two", ["2", "dos"]) fun test_one_two(one, two): ...
pytest.fixture - You can't pass arguments to other fixtures with
pytest.fixture. There's noindirectkeyword
It would be nice if the two syntaxes were unified
If I may suggest something, here's my take on a better syntax. Most of it is already possible with pytest; here's a runnable proof of concept — see below regarding what it can't do.
You can define fixture like this:
from ... import fixture @fixture def dog(request, /, name, age=69): return f"{name} the dog aged {age}"Here, to the left of / you have other fixtures, and to the right you have parameters that are supplied using:
@dog.arguments("Buddy", age=7) def test_with_dog(dog): assert dog == "Buddy the dog aged 7"Note: This works the same way function arguments work. If you don't supply the
ageargument, the default one,69, is used instead. if you don't supplyname, or omit thedog.argumentsdecorator, you get the regularTypeError: dog() missing 1 required positional argument: 'name'. If you have another fixture that takes argumentname, it doesn't conflict with this one.
Also note:
/is not strictly required, but it allows easily building this on top of existing pytest api, and also should prevent a few user errors
If for some reason you can't import dog directly, you can reference it by name:
dog = fixture.by_name("dog")To run a test several times with different dogs, use:
from ... import arguments @dog.argumentize(arguments("Champion"), arguments("Buddy", age=7)) def test_with_dog(dog): ...Note: I'm using “argumentize” here for a lack of a better word. Pytest documentation says,
The builtin pytest.mark.parametrize decorator enables parametrization of arguments for a test function.
But arguments (e.g.
"Champion") are not the subject but the object of this action. I'm not sure if “parametrize” fits here; the testtest_with_dogis already parametrized by virtue of having the parameterdog. The subject must be the parameter here, so perhaps this should say “argumentization of parameters”? (Disclaimer: English isn't my mother tongue)
...it would be reasonable to not require arguments() in case of a single positional argument:
@dog.argumentize("Champion", arguments("Buddy", age=7)) def test_with_dog(dog): ...To pass parameters to another fixture, stack arguments() decorators:
@cat.arguments("Mittens") @dog.arguments("Buddy", age=7) def test_with_cat_and_dog(cat, dog): # this test is run once ...To run a matrix of tests, stack argumentize() decorators:
@cat.argumentize("Tom", "Mittens") @dog.argumentize("Champion", arguments("Buddy", age=7)) def test_with_cat_and_dog(cat, dog): # this test is run four times ...To argumentize several parameters as a group, use this:
from ... import parameters @parameters(dog, cat).argumentize( ("Champion", arguments("Tom", age=420)), ("Buddy", "Whiskers") ) def test_with_dogs_and_cats(dog, cat): # this test is run twice ...To argumentize a parameter directly, without passing arguments to other fixtures, use a special “literal” fixture:
@fixture.literal("weasel").argumentize("Bob", "Mike") def test_with_weasel(weasel): assert weasel in ["Bob", "Mike"]This fixture works exactly like regular fixtures. In parameters(), you can omit the call to fixture.literal():
@parameters("expression", "result").argumentize( ("1 + 1", 2), ("2 * 2", 4), ) def test_math(expression, result): assert eval(expression) == resultYou can mix it all together. Also, you can use pytest.param() as usual:
@parameters(dog, owner, "poop").argumentize( (arguments("Champion", age=1337), "Macaulay Culkin", "brown"), ("Buddy", arguments(), "round"), pytest.param("Buddy", "Name", 123, marks=pytest.mark.xfail()) ) def test_with_dogs_and_owners_and_poop(dog, owner, poop): assert f"{dog=}, {owner=}, {poop=}" in [ "dog='Champion the dog aged 1337', owner='Macaulay Culkin, owner of Champion the dog aged 1337', poop='brown'", "dog='Buddy the dog aged 69', owner='John Doe, owner of Buddy the dog aged 69', poop='round'", "dog='Buddy the dog aged 69', owner='Name, owner of Buddy the dog aged 69', poop=123" ] assert isinstance(poop, str)Pass a keyword argument ids to argumentize() to have readable test case descriptions. I think you can't do this better than pytest is doing it at the time being.
You can also parametrize fixtures using the same syntax, including parameter matrices, etc
@fixture.literal("name").argumentize("Veronica", "Greta") @fixture.literal("word").argumentize("boo", "meow") @fixture def hedgehog(self, request, /, name, word): return f"{name} the hedgehog says: {word}" def test_with_hedgehog(self, hedgehog): # this test is run twice assert hedgehog in [ "Veronica the hedgehog says: boo", "Greta the hedgehog says: boo", "Veronica the hedgehog says: meow", "Greta the hedgehog says: meow", ]Notes on my proof of concept:
- With the exception of reading and mutating
_pytestfixturefunction, it uses the regular pytest api. It doesn't require any breaking changes. It would require very few changes in existing pytest code if implemented properly. - It is alrady usable, and also supports async fixtures. Also it produces a neat setup plan:
$ pytest --setup-plan SETUP F dog['Buddy', age=7] ... SETUP F dog['Champion'] SETUP F owner (fixtures used: dog)['John Travolta'] ... - It doesn't support
ids, although it could. Matricizing fixture function arguments is a bit hacky and throwingidsinto that would complicate things too much for a proof of concept - When argumentizing fixture functions, you can only use literal parameters, as pytest doesn't provide syntax to pass arguments to other fixtures in
@pytest.fixture() - When matricizing fixture function parameters, tuples of marks are simply merged into one. This probably isn't right but works for the purpose of this POC
- In the case you write the following, which you shouldn't do, the test will fail: Instead,
@fixture.literal("foo").argumentize(arguments("bar")) def test_with_foo(foo): assert foo == "bar"
foowill be== Arguments("bar"). I'm not sure if one other other behavior should be preferred here