Add a function to detect if pytest is running

[adamchainz]

What's the problem this feature will solve?

Django settings modules often load environment variables from a .env file:

from pathlib import Path

import dotenv

BASE_DIR = Path(__file__).resolve().parent.parent

dotenv.load_dotenv(dotenv_path=BASE_DIR / ".env")

Loading from .env is not desirable during test runs. Environment variables added for local debugging may affect the test run in unexpected ways, such as making remote API calls.

The settings module should therefore only load the .env file if pytest isn't running:

from pathlib import Path

import dotenv

BASE_DIR = Path(__file__).resolve().parent.parent

if ... # pytest is not running
    dotenv.load_dotenv(dotenv_path=BASE_DIR / ".env")

Changing behaviour during tests is usually undesirable, but here it's done to isolate the test run, which is a good thing.

pytest documents a pattern to change behaviour using fixtures here: https://docs.pytest.org/en/latest/example/simple.html#detect-if-running-from-within-a-pytest-run . Unfortunately this techinque is incompatible with this particular problem. pytest-django's fixtures load the settings module early, before fixtures from conftest.py fixtures run, so the pattern cannot be applied. And environment variable changes cannot be undone after the fact.

To solve this I made pytest-is-running, which can be used like so:

from pathlib import Path

import dotenv
import pytest_is_running

BASE_DIR = Path(__file__).resolve().parent.parent

if not pytest_is_running.is_running():
    dotenv.load_dotenv(dotenv_path=BASE_DIR / ".env")

pytest-is-running works because plugin fixtures can load early, and it uses hooks with tryfirst=True.

Describe the solution you'd like

Rather than have a documented workaround and plugin, I'd like to see a function in core pytest that provides the ability to check if pytest is running. It could simplify things for those using the both the documented pattern and the plugin.

It would also be good if the solution could somehow avoid the cost of doing import pytest, so that non-test pathways do not import the heap of stuff in pytest only to ignore it because pytest isn't running. The plugin is deliberately structured to avoid this.

Alternative Solutions

I wrote the plugin.

Additional context

none

[The-Compiler]

We currently just document how to do this, probably in a way similarly to what your plugin does.

Note you could also just check if "pytest" in sys.modules: as long as your code under test does not import pytest.

We do have a PYTEST_CURRENT_TEST environment variable already. Personally I'd be fine with having a PYTEST_VERSION or whatever which gets set very early, I suppose PYTEST_CURRENT_TEST will be set too late for it to be useful at import time.

[The-Compiler]

Also see pytest-dev/pytest-django#333 which has some additional considerations and use-cases around this.

[nicoddemus]

Personally I'd be fine with having a PYTEST_VERSION or whatever which gets set very early,

I'm OK with this solution. It would also remove that env var from os.environ at the end of pytest.main(), I assume?

[bluetech]

A while ago I had an idle thought that something like this should be a python feature. I was thinking a builtin constant __test__, similar to __debug__, that would normally be false, but that a test runner would set to true while discovering and running tests.

In the Rust language for example it is very common to interleave unit tests with the code they're testing in the same file using an attribute. In Python this would be:

def fibonacci(n): ...

if __test__:
    import pytest

    def test_fibonacci(n):
        assert fibonacci(10) == 55

        with pytest.raises(ValueError):
            fibonacci(-1)

def factorial(n): ...

if __test__:
    def test_factorial(n):
        assert factorial(50) == 30414093201713378043612608166064768844377641568960512000000000000

Not for everyone, but I think there are cases I would have used this.

[RonnyPfannschmidt]

after the drop of python 3.6 we can use https://docs.python.org/3/library/contextvars.html

i would propose to use this for gathering whether pytest is running

i woudl strictly oppose exposing sometihng like config as a contextvar however unless we had a stack for nested pytest session

[adamchainz]

We currently just document how to do this, probably in a way similarly to what your plugin does.

Yes, I noted in the description why this doesn't work for the case of the django settings file 😊

---

I like the idea of the PYTEST_VERSION (or whatever) environment variable. This is general and wouldn't require import pytest in production code. Yes it should be popped out of os.environ after pytest completes its run.

A __test__ builtin would be interesting, but it wouldn't be so easy to add. If it was only set by pytest, it wouldn't exist without importing pytest, unless it were actually added to Python, and placed in a backports package.

contextvars seem like they'd also require import pytest to use, since the var must be defined in a module—unless pytest shipped a side module to contain the var?

[RonnyPfannschmidt]

One can use sys.modules.get in such helpers

Personally im -1 on adding easy and obvious fast paths to make it easy to keep messy global configuration around

This stems from stuff going horrendously wrong every time a project uses that and then suddenly some tests need to run with different configurations or multiple configurations

[adamchainz]

One can use sys.modules.get in such helpers

🤔 Idk why I didn't think of that, thanks for the hint.

I think for now I will try "pytest" in sys.modules instead of my plugin.

Agree that it can go wrong as complexity creeps in.

[Bengt]

I stopped my overengineering when I arrived at:

def we_are_running_via_pytest() -> bool:
    """Detect if we are running via pytest."""
    # When we are running via pytest it is loaded to the sys modules dictionary.
    _we_are_running_via_pytest: bool = 'pytest' in sys.modules
    return _we_are_running_via_pytest

Source: https://docs.python.org/3/library/sys.html#sys.modules

Use it like so:

def get_configuration() -> Configuration:
    configuration: Configuration

    if we_are_running_via_pytest():
        print('Creating test configuration.')
        configuration = _get_test_configuration()
    else:
        print('Creating runtime configuration.')
        configuration = _get_runtime_configuration()

    return configuration

[nicoddemus]

Thanks everyone!

I'm closing this for now, with the recommended solution being:

is_running_pytest = "pytest" in sys.modules

If somebody wants to contribute that to the documentation, it would be great!