pathlib: Path('.').exists() returns True when current working directory (cwd) was deleted

[dan-blanchard]

Bug report

Bug description:

Path.exists() and Path.resolve() do not work consistently with each other if the current working directory is deleted after Python is launched.

For example, if I open a Python 3.13.0 interpreter (although I have verified this occurs in older versions as well) in a directory, and then delete that directory I experience the following behavior:

>>> from pathlib import Path
>>> Path('.').exists()
True
>>> Path('.').resolve()
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    Path('.').resolve()
    ~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/Caskroom/mambaforge/base/envs/py313minimal/lib/python3.13/pathlib/_local.py", line 670, in resolve
    return self.with_segments(os.path.realpath(self, strict=strict))
                              ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 413, in realpath
FileNotFoundError: [Errno 2] No such file or directory

Note how Path('.').exists() is True, even though the cwd does not exist, and then when I try to resolve the path, I get a FileNotFoundError.

You also get a FileNotFoundError when the cwd doesn't exist for all relative paths, even though in the non-dot cases Path.exists() works as one would expect.

>>> Path('asdf').exists()
False
>>> Path('asdf').resolve().exists()
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    Path('asdf').resolve().exists()
    ~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/Caskroom/mambaforge/base/envs/py313minimal/lib/python3.13/pathlib/_local.py", line 670, in resolve
    return self.with_segments(os.path.realpath(self, strict=strict))
                              ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 413, in realpath
FileNotFoundError: [Errno 2] No such file or directory

If the cwd does exist, you can see that Path.resolve() does not raise a FileNotFoundError when the path does not exist.

>>> from pathlib import Path
>>> Path('asdf').exists()
False
>>> Path('asdf').resolve()
PosixPath('/Users/danielblanchard/asdf')

I believe resolve is doing the right thing here, but two things are not working as expected when the cwd is deleted:

1. Path('.').exists() and Path('').exists() should not tell you a path exists that doesn't.
2. Path('foo').resolve() should not raise a FileNotFoundError if the cwd doesn't exist, unless strict=True. Instead it should just return the initial path.

CPython versions tested on:

3.9, 3.12, 3.13

Operating systems tested on:

Linux, macOS

[barneygale]

I believe this is reproducible with os.path.exists() and os.path.realpath() too. Currently exists() returns true if stat() succeeds, and unlinking the path from the filesystem isn't enough to make stat() fail on POSIX - https://unix.stackexchange.com/questions/434417/what-happens-when-the-current-directory-is-deleted

[ronaldoussoren]

I believe resolve is doing the right thing here, but two things are not working as expected when the cwd is deleted:

1. Path('.').exists() and Path('').exists() should not tell you a path exists that doesn't.

On unix systems when the current working directory (CWD) is removed the directory still exists, but is "just" unlinked from the directory tree and marked as unmodifiable (as in, you cannot create new entries in the directory).

So, IMHO Path(".").exists() returning True is not a bug.

2. Path('foo').resolve() should not raise a FileNotFoundError if the cwd doesn't exist, unless strict=True. Instead it should just return the initial path.

The API documentation for the resolve method says it returns an absolute path. Path("foo").resolve() cannot return an absolute path when the CWD is removed, there no longer is a valid path from the filesystem root to the CWD. IMHO the current behaviour is not a bug, although one could argue that the documentation could be more explicit about this.

[dan-blanchard]

I can see that both of those things have logical reasons for being true, but it is very confusing as a user when you write something like:

dot_path = Path(".")
if dot_path.exists():
    dot_path = dot_path.resolve()

and it raises a FileNotFoundError. At the very least, docs that mention that checking if the path exists isn't good enough to prevent a FileNotFoundError even when strict=False would be helpful.

[ronaldoussoren]

I can see that both of those things have logical reasons for being true, but it is very confusing as a user when you write something like:

dot_path = Path(".")
if dot_path.exists():
    dot_path = dot_path.resolve()

and it raises a FileNotFoundError. At the very least, docs that mention that checking if the path exists isn't good enough to prevent a FileNotFoundError even when strict=False would be helpful.

Checking before resolving isn't good enough in general, the filesystem might change between the check and the call to resolve. This is a well known source of security issues in other use cases (although that shouldn't be relevant here).

[barneygale]

Any suggestions on how we might be able to work around this in pathlib? Running resolve() in exists() is way too costly just to fix this I'm afraid.

[ronaldoussoren]

Any suggestions on how we might be able to work around this in pathlib? Running resolve() in exists() is way too costly just to fix this I'm afraid.

IMHO this is a documentation issue at best and doesn't need to be worked around. On Unix systems an empty directory can be removed from the filesystem tree while still being used (as a CWD or opened using O_DIRECTORY). Those directories still exist, but cannot be resolved in an absolute path. That's similar to how an open file can be removed, and then still can be used even if not present in the directory tree.

[dan-blanchard]

I agree that this is mostly a documentation issue. It is unexpected behavior, but with the explanation from @ronaldoussoren it makes sense to me why this is the case. If the docs for exists made it clear that it will return True on POSIX for the CWD when the CWD has been unlinked/deleted, that would be helpful.

It would also be useful to have a more general warning on exists that you shouldn't use it to check if things exist before resolving for the security reasons @ronaldoussoren referenced.