Description
Bug description
The change in d282424 has significantly altered the way parametrized fixtures are cached by changing the operator used to compare them: using is instead of __eq__. For fixtures that are parametrized using strings (and potentially other parameter types) this has caused strange (read: unexpected) behavior where fixtures are recreated when the user would expect them to be cached and re-used. Since re-use of expensive-to-set-up fixtures is precisely why fixtures exists, I believe this is bug and that the change above should be reverted or at least adapted.
This bug obviously doesn't affect pytest versions without the commit d282424; i.e. pytest < 5.4.0.
Example
The behavior described above can be illustrated by the following code:
import pytest
def pytest_generate_tests(metafunc):
if "my_fixture" in metafunc.fixturenames:
param = "d%s" % "1"
print("\nparam id=%d" % id(param), flush=True)
metafunc.parametrize("my_fixture", [param, "d2"], indirect=True)
@pytest.fixture(scope='session')
def my_fixture(request):
print("Setting up fixture scope=session ", end='')
def test1(my_fixture):
pass
def test2(my_fixture):
pass
The result of running this with pyrest 5.4.1 is:
$ pytest -v -s test_pytest.py
platform darwin -- Python 3.6.8, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- <redacted path>
cachedir: .pytest_cache
rootdir: <redacted path>
plugins: testinfra-3.2.0, custom-exit-code-0.3.0, flaky-3.6.1, ordering-0.6, container-tests-1.0
collecting ...
param id=4409052776
param id=4408955544
collected 4 items
test_pytest.py::test1[d1] Setting up fixture scope=session PASSED
test_pytest.py::test2[d1] Setting up fixture scope=session PASSED
test_pytest.py::test1[d2] Setting up fixture scope=session PASSED
test_pytest.py::test2[d2] PASSED
Note how the fixture is created 3 times and not 2 times.
Now compare this with the following slightly modified code (the only difference is the way the value of param is created in pytest_generate_tests ):
import pytest
def pytest_generate_tests(metafunc):
if "my_fixture" in metafunc.fixturenames:
param = "d1"
print("\nparam id=%d" % id(param), flush=True)
metafunc.parametrize("my_fixture", [param, "d2"], indirect=True)
@pytest.fixture(scope='session')
def my_fixture(request):
print("Setting up fixture scope=session ", end='')
def test1(my_fixture):
pass
def test2(my_fixture):
pass
pytest -v -s test_pytest.py
================================================================================================ test session starts =================================================================================================
platform darwin -- Python 3.6.8, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- <redacted path>
cachedir: .pytest_cache
rootdir: <redacted path>
plugins: testinfra-3.2.0, custom-exit-code-0.3.0, flaky-3.6.1, ordering-0.6, container-tests-1.0
collecting ...
param id=4583559776
param id=4583559776
collected 4 items
test_pytest.py::test1[d1] Setting up fixture scope=session PASSED
test_pytest.py::test2[d1] PASSED
test_pytest.py::test1[d2] Setting up fixture scope=session PASSED
test_pytest.py::test2[d2] PASSED
Note how the fixture is now, as one would expected, instantiated only twice.
Conclusion
I think it's dangerous to use is to compare the values of objects since it compares addresses, as illustrated above. Using is might be the right thing to do for Numpy arrays but not for the general case. As a result I suggest reverting the change d282424, maybe using is to compare Numpy arrays specifically and continuing to use __eq__ for all other objects.