User site packages are being added when site module is enabled in _pth file

[tanzim]

Bug report

Bug description:

See #119920 (comment).

I've managed to reproduce this with the latest official Windows embedded Python releases for 3.11 and 3.12.

CPython versions tested on:

3.11, 3.12

Operating systems tested on:

Windows

Linked PRs

• gh-120037: User site packages are being added when site module is enabled in _pth file #120100

[eryksun]

This is a straight-forward fix. Add the statement config['user_site_directory'] = 0 in the following section:

cpython/Modules/getpath.py

Lines 760 to 764 in 69b3e8e

	if pth:
	config['isolated'] = 1
	config['use_environment'] = 0
	config['site_import'] = 0
	config['safe_path'] = 1

[tanzim]

While reproducing this, I noted something else on macOS - this is assuming Python 3.11 adds support for ._pth file on macOS:

1. Placing ._pth file with interpreter executable name in the interpreter dir, e.g. bin works
2. Enabling site has the same issue as above
3. However, no matter how much I tried I couldn't get a .pth file to work, doesn't matter where I placed it.

[eryksun]

However, no matter how much I tried I couldn't get a .pth file to work, doesn't matter where I placed it.

On Windows, the site module executes .pth files found in the prefix directory; the site-packages directory "{prefix}\Lib\site-packages"; and, if enabled, the user site-packages directory. On POSIX, it checks the site-packages directory and, if enabled, the user site-packages directory, but not the prefix directory. The path of the site-packages directory on POSIX is usually of the form "{prefix}/lib/pythonX.Y/site-packages", unless a distribution customizes it.

After adding the site-packages and user site-packages directories, and executing their .pth files, the site module tries to import a sitecustomize module. This may be simpler for you to manage since it doesn't matter where sitecustomize is located.

[tanzim]

On POSIX, it checks the site-packages directory and, if enabled, the user site-packages directory, but not the prefix directory.

That explains why the .pth file didn't work for me same way as Windows even though I tried placing it in prefix, prefix/bin and site packages as above. I can confirm sitecustomize works. I have to say though, the various initialization logic is pretty convoluted!

Thanks again for the help.

[tanzim]

Could be another esoteric piece of information that I lack however I noted that when using sitecustomize on macOS, even though import site in defined in the ._pth file, site-packages directory, e.g. /lib/python3.11/site-packages is not added to sys.path by default. On Windows it is. I could of course manually add it in the ._pth file but that feels hacky.

[eryksun]

Could be another esoteric piece of information that I lack however I noted that when using sitecustomize on macOS, even though import site in defined in the ._pth file, site-packages directory, e.g. /lib/python3.11/site-packages is not added to sys.path by default. On Windows it is. I could of course manually add it in the ._pth file but that feels hacky.

Maybe the value of sys.prefix is wrong. I don't use macOS, so I can't test this.

[tanzim]

You guessed right. It's pointing to python/bin/lib/python3.11/site-packages and sys.prefix is set to python/bin when it should be one level up.

I'm pretty sure this is a side effect of colocating the ._pth file with the python executable in the bin folder. If I remove the _.pth sys.prefix works as expected and site.getsitepackages() returns the correct value.