Skip to content

gh-88569: add ntpath.isreserved() #95486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 38 commits into from
Jan 26, 2024
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
6b89ecc
gh-88569: add `os.path.isreserved()`
barneygale Jul 31, 2022
3e08e4a
Fix tests
barneygale Jul 31, 2022
49ba439
Remove implementation in `genericpath`
barneygale Jul 31, 2022
238b3e3
Apply suggestions from code review
barneygale Aug 9, 2022
f647502
Apply suggestions from code review
barneygale Aug 10, 2022
f07c7ad
Remove tests for pathlib.PurePath.is_reserved().
barneygale Aug 10, 2022
dc857c9
Speed up isreserved('.') and '..'
barneygale Aug 12, 2022
f0fd2c8
Note change to algorithm in pathlib docs.
barneygale Aug 12, 2022
3b51db8
Update Lib/ntpath.py
barneygale Aug 12, 2022
06cb428
Update Doc/library/os.path.rst
barneygale Aug 13, 2022
79c0be4
Update Lib/posixpath.py
barneygale Aug 13, 2022
0a0db6a
Restore `os.fspath()` call in `posixpath.isreserved()`
barneygale Aug 16, 2022
7145b86
Apply suggestions from code review
barneygale Aug 23, 2022
9f74b64
posixpath.isreserved(): return True for paths with NUL characters
barneygale Aug 23, 2022
91b2bb3
ntpath.isreserved(): minor tweaks
barneygale Aug 23, 2022
e6aff58
ntpath.isreserved(): restore initial splitdrive() call
barneygale Aug 23, 2022
e6a2c0b
ntpath.isreserved(): avoid calling `splitdrive()` repeatedly.
barneygale Aug 23, 2022
14dde15
Update Lib/ntpath.py
barneygale Aug 23, 2022
fab274a
Add `isreservedname()` for discussion.
barneygale Aug 23, 2022
936dcc8
Apply suggestions from code review
barneygale Aug 23, 2022
3fb127f
Undo posixpath changes
barneygale Jan 8, 2024
002d951
Merge branch 'main' into os-path-isreserved
barneygale Jan 8, 2024
c8ed711
Update version numbers.
barneygale Jan 8, 2024
c772b25
Fix syntax, whitespace.
barneygale Jan 8, 2024
3fbef57
Make `isreservedname()` private
barneygale Jan 8, 2024
4b34274
Update Lib/ntpath.py
barneygale Jan 8, 2024
a877677
Tighten up `PurePath.is_reserved()` exception handling.
barneygale Jan 8, 2024
3721c8c
Use `str(self)` to support non-os.PathLike implementations.
barneygale Jan 8, 2024
b905d2f
Deprecate `pathlib.PurePath.is_reserved()`
barneygale Jan 8, 2024
2756ffb
Add note about approximate and changing Windows rules; remove doctest.
barneygale Jan 8, 2024
c03c672
Update Doc/library/os.path.rst
barneygale Jan 8, 2024
4085ff5
Update what's new.
barneygale Jan 8, 2024
b4b3d0b
Mention deprecation in NEWS
barneygale Jan 8, 2024
9e2d21f
Address review feedback
barneygale Jan 8, 2024
44c37cb
Point to `os.path.isreserved()` in whatsnew deprecation notices.
barneygale Jan 8, 2024
f9033b3
Merge branch 'main' into os-path-isreserved
barneygale Jan 8, 2024
efb7681
Merge branch 'main' into os-path-isreserved
barneygale Jan 14, 2024
e398c3f
Address review feedback
barneygale Jan 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions Doc/library/os.path.rst
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,27 @@ the :mod:`glob` module.)
.. versionadded:: 3.12


.. function:: isreserved(path)

Return ``True`` if *path* is a reserved pathname on the current system.

On Windows, reserved filenames include those that end with a space or dot;
those that contain colons (i.e. file streams such as "name:stream"),
wildcard characters (i.e. ``'*?"<>'``), pipe, or ASCII control characters;
as well as DOS device names such as "NUL", "CON", "CONIN$", "CONOUT$",
"AUX", "PRN", "COM1", and "LPT1".

.. note::

This function approximates rules for reserved paths on most Windows
systems. It may be updated in future Python releases as changed rules
become more widely used.

.. availability:: Windows.

.. versionadded:: 3.13


.. function:: join(path, *paths)

Join one or more path segments intelligently. The return value is the
Expand Down
10 changes: 4 additions & 6 deletions Doc/library/pathlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -541,13 +541,11 @@ Pure paths provide the following methods and properties:
reserved under Windows, ``False`` otherwise. With :class:`PurePosixPath`,
``False`` is always returned.

>>> PureWindowsPath('nul').is_reserved()
True
>>> PurePosixPath('nul').is_reserved()
False
.. versionchanged:: 3.13
Windows path names that contain a colon, or end with a dot or a space,
are considered reserved. UNC paths may be reserved.

File system calls on reserved paths can fail mysteriously or have
unintended effects.
.. deprecated-removed:: 3.13 3.15


.. method:: PurePath.joinpath(*pathsegments)
Expand Down
41 changes: 39 additions & 2 deletions Lib/ntpath.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
__all__ = ["normcase","isabs","join","splitdrive","splitroot","split","splitext",
"basename","dirname","commonprefix","getsize","getmtime",
"getatime","getctime", "islink","exists","lexists","isdir","isfile",
"ismount", "expanduser","expandvars","normpath","abspath",
"curdir","pardir","sep","pathsep","defpath","altsep",
"ismount","isreserved","expanduser","expandvars","normpath",
"abspath","curdir","pardir","sep","pathsep","defpath","altsep",
"extsep","devnull","realpath","supports_unicode_filenames","relpath",
"samefile", "sameopenfile", "samestat", "commonpath", "isjunction"]

Expand Down Expand Up @@ -337,6 +337,43 @@ def ismount(path):
return False


_reserved_chars = frozenset(
{chr(i) for i in range(32)} |
{'"', '*', ':', '<', '>', '?', '|', '/', '\\'}
)

_reserved_names = frozenset(
{'CON', 'PRN', 'AUX', 'NUL', 'CONIN$', 'CONOUT$'} |
{f'COM{c}' for c in '123456789\xb9\xb2\xb3'} |
{f'LPT{c}' for c in '123456789\xb9\xb2\xb3'}
)

def isreserved(path):
"""Return true if the pathname is reserved by the system."""
# Refer to "Naming Files, Paths, and Namespaces":
# https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file
path = os.fsdecode(splitroot(path)[2]).replace(altsep, sep)
return any(_isreservedname(name) for name in reversed(path.split(sep)))

def _isreservedname(name):
"""Return true if the filename is reserved by the system."""
name = os.fsdecode(name)
# Trailing dots and spaces are reserved.
if name.endswith(('.', ' ')) and name not in ('.', '..'):
return True
# Wildcards, separators, colon, and pipe (*?"<>/\:|) are reserved.
# ASCII control characters (0-31) are reserved.
# Colon is reserved for file streams (e.g. "name:stream[:type]").
if _reserved_chars.intersection(name):
return True
# DOS device names are reserved (e.g. "nul" or "nul .txt"). The rules
# are complex and vary across Windows versions. On the side of
# caution, return True for names that may not be reserved.
if name.partition('.')[0].rstrip(' ').upper() in _reserved_names:
return True
return False


# Expand paths beginning with '~' or '~user'.
# '~' means $HOME; '~user' means that user's home directory.
# If the path doesn't begin with '~', or if the user or $HOME is unknown,
Expand Down
10 changes: 10 additions & 0 deletions Lib/pathlib/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,16 @@ def is_relative_to(self, other, /, *_deprecated):
other = self.with_segments(other, *_deprecated)
return _abc.PurePathBase.is_relative_to(self, other)

def is_reserved(self):
"""Return True if the path contains one of the special names reserved
by the system, if any."""
msg = ("pathlib.PurePath.is_reserved() is deprecated and "
"scheduled for removal in Python 3.15")
warnings.warn(msg, DeprecationWarning, stacklevel=2)
if self.pathmod is ntpath:
return self.pathmod.isreserved(self)
return False

def as_uri(self):
"""Return the path as a URI."""
if not self.is_absolute():
Expand Down
24 changes: 0 additions & 24 deletions Lib/pathlib/_abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,6 @@
# Maximum number of symlinks to follow in PathBase.resolve()
_MAX_SYMLINKS = 40

# Reference for Windows paths can be found at
# https://learn.microsoft.com/en-gb/windows/win32/fileio/naming-a-file .
_WIN_RESERVED_NAMES = frozenset(
{'CON', 'PRN', 'AUX', 'NUL', 'CONIN$', 'CONOUT$'} |
{f'COM{c}' for c in '123456789\xb9\xb2\xb3'} |
{f'LPT{c}' for c in '123456789\xb9\xb2\xb3'}
)

_WINERROR_NOT_READY = 21 # drive exists but is not accessible
_WINERROR_INVALID_NAME = 123 # fix for bpo-35306
_WINERROR_CANT_RESOLVE_FILENAME = 1921 # broken symlink pointing to itself
Expand Down Expand Up @@ -448,22 +440,6 @@ def is_absolute(self):
else:
return self.pathmod.isabs(str(self))

def is_reserved(self):
"""Return True if the path contains one of the special names reserved
by the system, if any."""
if self.pathmod is posixpath or not self._tail:
return False

# NOTE: the rules for reserved names seem somewhat complicated
# (e.g. r"..\NUL" is reserved but not r"foo\NUL" if "foo" does not
# exist). We err on the side of caution and return True for paths
# which are not considered reserved by Windows.
if self.drive.startswith('\\\\'):
# UNC paths are never reserved.
return False
name = self._tail[-1].partition('.')[0].partition(':')[0].rstrip(' ')
return name.upper() in _WIN_RESERVED_NAMES

def match(self, path_pattern, *, case_sensitive=None):
"""
Return True if this path matches the given pattern.
Expand Down
56 changes: 56 additions & 0 deletions Lib/test/test_ntpath.py
Original file line number Diff line number Diff line change
Expand Up @@ -973,6 +973,62 @@ def test_ismount(self):
self.assertTrue(ntpath.ismount(b"\\\\localhost\\c$"))
self.assertTrue(ntpath.ismount(b"\\\\localhost\\c$\\"))

def test_isreserved(self):
self.assertFalse(ntpath.isreserved(''))
self.assertFalse(ntpath.isreserved('.'))
self.assertFalse(ntpath.isreserved('..'))
self.assertFalse(ntpath.isreserved('/'))
self.assertFalse(ntpath.isreserved('/foo/bar'))
# A name that ends with a space or dot is reserved.
self.assertTrue(ntpath.isreserved('foo.'))
self.assertTrue(ntpath.isreserved('foo '))
# ASCII control characters are reserved.
self.assertTrue(ntpath.isreserved('\foo'))
# Wildcard characters, colon, and pipe are reserved.
self.assertTrue(ntpath.isreserved('foo*bar'))
self.assertTrue(ntpath.isreserved('foo?bar'))
self.assertTrue(ntpath.isreserved('foo"bar'))
self.assertTrue(ntpath.isreserved('foo<bar'))
self.assertTrue(ntpath.isreserved('foo>bar'))
self.assertTrue(ntpath.isreserved('foo:bar'))
self.assertTrue(ntpath.isreserved('foo|bar'))
# Case-insensitive DOS-device names are reserved.
self.assertTrue(ntpath.isreserved('nul'))
self.assertTrue(ntpath.isreserved('aux'))
self.assertTrue(ntpath.isreserved('prn'))
self.assertTrue(ntpath.isreserved('con'))
self.assertTrue(ntpath.isreserved('conin$'))
self.assertTrue(ntpath.isreserved('conout$'))
# COM/LPT + 1-9 or + superscript 1-3 are reserved.
self.assertTrue(ntpath.isreserved('COM1'))
self.assertTrue(ntpath.isreserved('LPT9'))
self.assertTrue(ntpath.isreserved('com\xb9'))
self.assertTrue(ntpath.isreserved('com\xb2'))
self.assertTrue(ntpath.isreserved('lpt\xb3'))
# DOS-device name matching ignores characters after a dot or
# a colon and also ignores trailing spaces.
self.assertTrue(ntpath.isreserved('NUL.txt'))
self.assertTrue(ntpath.isreserved('PRN '))
self.assertTrue(ntpath.isreserved('AUX .txt'))
self.assertTrue(ntpath.isreserved('COM1:bar'))
self.assertTrue(ntpath.isreserved('LPT9 :bar'))
# DOS-device names are only matched at the beginning
# of a path component.
self.assertFalse(ntpath.isreserved('bar.com9'))
self.assertFalse(ntpath.isreserved('bar.lpt9'))
# The entire path is checked, except for the drive.
self.assertTrue(ntpath.isreserved('c:/bar/baz/NUL'))
self.assertTrue(ntpath.isreserved('c:/NUL/bar/baz'))
self.assertFalse(ntpath.isreserved('//./NUL'))
# Bytes are supported.
self.assertFalse(ntpath.isreserved(b''))
self.assertFalse(ntpath.isreserved(b'.'))
self.assertFalse(ntpath.isreserved(b'..'))
self.assertFalse(ntpath.isreserved(b'/'))
self.assertFalse(ntpath.isreserved(b'/foo/bar'))
self.assertTrue(ntpath.isreserved(b'foo.'))
self.assertTrue(ntpath.isreserved(b'nul'))

def assertEqualCI(self, s1, s2):
"""Assert that two strings are equal ignoring case differences."""
self.assertEqual(s1.lower(), s2.lower())
Expand Down
50 changes: 8 additions & 42 deletions Lib/test/test_pathlib/test_pathlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,12 @@ def test_is_relative_to_several_args(self):
with self.assertWarns(DeprecationWarning):
p.is_relative_to('a', 'b')

def test_is_reserved_deprecated(self):
P = self.cls
p = P('a/b')
with self.assertWarns(DeprecationWarning):
p.is_reserved()


class PurePosixPathTest(PurePathTest):
cls = pathlib.PurePosixPath
Expand Down Expand Up @@ -287,13 +293,6 @@ def test_is_absolute(self):
self.assertTrue(P('//a').is_absolute())
self.assertTrue(P('//a/b').is_absolute())

def test_is_reserved(self):
P = self.cls
self.assertIs(False, P('').is_reserved())
self.assertIs(False, P('/').is_reserved())
self.assertIs(False, P('/foo/bar').is_reserved())
self.assertIs(False, P('/dev/con/PRN/NUL').is_reserved())

def test_join(self):
P = self.cls
p = P('//a')
Expand Down Expand Up @@ -951,41 +950,6 @@ def test_div(self):
self.assertEqual(p / P('./dd:s'), P('C:/a/b/dd:s'))
self.assertEqual(p / P('E:d:s'), P('E:d:s'))

def test_is_reserved(self):
P = self.cls
self.assertIs(False, P('').is_reserved())
self.assertIs(False, P('/').is_reserved())
self.assertIs(False, P('/foo/bar').is_reserved())
# UNC paths are never reserved.
self.assertIs(False, P('//my/share/nul/con/aux').is_reserved())
# Case-insensitive DOS-device names are reserved.
self.assertIs(True, P('nul').is_reserved())
self.assertIs(True, P('aux').is_reserved())
self.assertIs(True, P('prn').is_reserved())
self.assertIs(True, P('con').is_reserved())
self.assertIs(True, P('conin$').is_reserved())
self.assertIs(True, P('conout$').is_reserved())
# COM/LPT + 1-9 or + superscript 1-3 are reserved.
self.assertIs(True, P('COM1').is_reserved())
self.assertIs(True, P('LPT9').is_reserved())
self.assertIs(True, P('com\xb9').is_reserved())
self.assertIs(True, P('com\xb2').is_reserved())
self.assertIs(True, P('lpt\xb3').is_reserved())
# DOS-device name mataching ignores characters after a dot or
# a colon and also ignores trailing spaces.
self.assertIs(True, P('NUL.txt').is_reserved())
self.assertIs(True, P('PRN ').is_reserved())
self.assertIs(True, P('AUX .txt').is_reserved())
self.assertIs(True, P('COM1:bar').is_reserved())
self.assertIs(True, P('LPT9 :bar').is_reserved())
# DOS-device names are only matched at the beginning
# of a path component.
self.assertIs(False, P('bar.com9').is_reserved())
self.assertIs(False, P('bar.lpt9').is_reserved())
# Only the last path component matters.
self.assertIs(True, P('c:/baz/con/NUL').is_reserved())
self.assertIs(False, P('c:/NUL/con/baz').is_reserved())


class PurePathSubclassTest(PurePathTest):
class cls(pathlib.PurePath):
Expand Down Expand Up @@ -1020,6 +984,8 @@ def tempdir(self):

def test_matches_pathbase_api(self):
our_names = {name for name in dir(self.cls) if name[0] != '_'}
# is_reserved() is deprecated and PurePath/Path-only
our_names.remove('is_reserved')
path_names = {name for name in dir(pathlib._abc.PathBase) if name[0] != '_'}
self.assertEqual(our_names, path_names)
for attr_name in our_names:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add :func:`ntpath.isreserved`, which identifies reserved pathnames on
Windows; reserved names include "NUL", "AUX" and "CON".