-
-
Notifications
You must be signed in to change notification settings - Fork 32.1k
gh-133164: Add PyUnstable_Object_IsUniqueReferencedTemporary
C API
#133170
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 4 commits
32b6cd0
479f185
4a53a98
fdc438a
b6508df
07ba540
99e3265
0e0e674
07f3b23
bba589f
f23de33
6efe2fb
f30ccf4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -613,6 +613,31 @@ Object Protocol | |
|
||
.. versionadded:: 3.14 | ||
|
||
.. c:function:: int PyUnstable_Object_IsUniqueTemporary(PyObject *obj) | ||
|
||
Check if *obj* is a unique temporary object on the top most frame of the | ||
interpreter stack. Returns ``1`` if *obj* is a unique temporary object, | ||
colesbury marked this conversation as resolved.
Show resolved
Hide resolved
|
||
and ``0`` otherwise. This function cannot fail, but the check is | ||
conservative, and may return ``0`` in some cases even if *obj* is a unique | ||
temporary object. | ||
|
||
If an object is a unique temporary, it is guaranteed that the reference | ||
count is ``1`` and it may be safe to modify the object in-place becuase | ||
colesbury marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think "may be safe" doesn't quite convey what we want here. We don't want people to be in doubt about whether it's safe from the interpreter's point of view. Maybe something along the lines of "it is guaranteed that the current code owns the only reference to the object"? It's probably also worth mentioning why a refcount of 1 isn't enough to guarantee it's a unique temporary, something about the interpreter borrowing references internally, similar to the whatsnew entry. |
||
it is not visible to any other code. | ||
ZeroIntensity marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
In the example below, ``my_func`` is called with a unique temporary object | ||
as its argument:: | ||
|
||
my_func([1, 2, 3]) | ||
|
||
In the example below, ``my_func`` is **not** called with a unique temporary | ||
object as its argument:: | ||
colesbury marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
my_list = [1, 2, 3] | ||
my_func(my_list) | ||
|
||
vstinner marked this conversation as resolved.
Show resolved
Hide resolved
|
||
.. versionadded:: 3.14 | ||
|
||
.. c:function:: int PyUnstable_IsImmortal(PyObject *obj) | ||
|
||
This function returns non-zero if *obj* is :term:`immortal`, and zero | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -88,6 +88,10 @@ If you encounter :exc:`NameError`\s or pickling errors coming out of | |
:mod:`multiprocessing` or :mod:`concurrent.futures`, see the | ||
:ref:`forkserver restrictions <multiprocessing-programming-forkserver>`. | ||
|
||
The interpreter avoids some reference count modifications internally when | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👌 |
||
it's safe to do so. This can lead to different values returned from | ||
:func:`sys.getrefcount` and :c:func:`Py_REFCNT` compared to previous versions | ||
of Python. See :ref:`below <whatsnew314-refcount>` for details. | ||
|
||
New features | ||
============ | ||
|
@@ -2135,6 +2139,11 @@ New features | |
take a C integer and produce a Python :class:`bool` object. (Contributed by | ||
Pablo Galindo in :issue:`45325`.) | ||
|
||
* Add :c:func:`PyUnstable_Object_IsUniqueTemporary` to determine if an object | ||
is a unique temporary object on the interpreter's operand stack. This can | ||
be used in some cases as a replacement for checking if :c:func:`Py_REFCNT` | ||
is ``1`` for Python objects passed as arguments to C API functions. | ||
|
||
|
||
Limited C API changes | ||
--------------------- | ||
|
@@ -2169,6 +2178,17 @@ Porting to Python 3.14 | |
a :exc:`UnicodeError` object. | ||
(Contributed by Bénédikt Tran in :gh:`127691`.) | ||
|
||
.. _whatsnew314-refcount: | ||
|
||
* The interpreter internally avoids some reference count modifications when | ||
loading objects onto the operands stack by :term:`borrowing <borrowed reference>` | ||
references when possible. This can lead to smaller reference count values | ||
compared to previous Python versions. C API extensions that checked | ||
:c:func:`Py_REFCNT` of ``1`` to determine if an function argument is not | ||
referenced by any other code should isntead use | ||
colesbury marked this conversation as resolved.
Show resolved
Hide resolved
|
||
:c:func:`PyUnstable_Object_IsUniqueTemporary` as a safer replacement. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The fallout of this is a bit wider than the need for this function, e.g. the Pandas check that broke was looking at if https://github.com/pandas-dev/pandas/blob/main/pandas/core/frame.py#L4156-L4161 (c.f. pandas-dev/pandas#61368). I'm actually not totally sure why Pandas was checking for In NumPy we switched all the refcount tests to check changes in refcounts before and after an operation rather than absolute refcount values. That works across Python versions. Maybe a separate paragraph advising against using the absolute values of reference counts in tests or runtime sanity checks? Could be a followup too. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, I think we should do another pass over this and the |
||
|
||
|
||
* Private functions promoted to public C APIs: | ||
|
||
* ``_PyBytes_Join()``: :c:func:`PyBytes_Join`. | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
Add :c:func:`PyUnstable_Object_IsUniqueTemporary` function for | ||
determining if an object exists as a unique temporary variable on the | ||
interpreter's stack. This is a replacement for some cases where checking | ||
that :c:func:`Py_REFCNT` is one is no longer sufficient to determine if it's | ||
safe to modify a Python object in-place with no visible side effects. |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,6 +15,7 @@ | |
#include "pycore_hamt.h" // _PyHamtItems_Type | ||
#include "pycore_initconfig.h" // _PyStatus_OK() | ||
#include "pycore_instruction_sequence.h" // _PyInstructionSequence_Type | ||
#include "pycore_interpframe.h" // _PyFrame_Stackbase() | ||
#include "pycore_list.h" // _PyList_DebugMallocStats() | ||
#include "pycore_long.h" // _PyLong_GetZero() | ||
#include "pycore_memoryobject.h" // _PyManagedBuffer_Type | ||
|
@@ -2616,6 +2617,35 @@ PyUnstable_Object_EnableDeferredRefcount(PyObject *op) | |
#endif | ||
} | ||
|
||
int | ||
PyUnstable_Object_IsUniqueTemporary(PyObject *op) | ||
{ | ||
if (!_PyObject_IsUniquelyReferenced(op)) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this just demonstrates that We need to check that the following are true:
I think There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's discuss the name for |
||
return 0; | ||
} | ||
|
||
_PyInterpreterFrame *frame = _PyEval_GetFrame(); | ||
if (frame == NULL) { | ||
return 0; | ||
} | ||
|
||
_PyStackRef *base = _PyFrame_Stackbase(frame); | ||
_PyStackRef *stackpointer = frame->stackpointer; | ||
int found = 0; | ||
colesbury marked this conversation as resolved.
Show resolved
Hide resolved
|
||
while (stackpointer > base) { | ||
stackpointer--; | ||
if (op == PyStackRef_AsPyObjectBorrow(*stackpointer)) { | ||
if (!PyStackRef_IsHeapSafe(*stackpointer)) { | ||
return 0; | ||
} | ||
found++; | ||
ZeroIntensity marked this conversation as resolved.
Show resolved
Hide resolved
colesbury marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
} | ||
|
||
// Check that we found exactly one reference to `op` | ||
return found == 1; | ||
colesbury marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
int | ||
PyUnstable_TryIncRef(PyObject *op) | ||
{ | ||
|
Uh oh!
There was an error while loading. Please reload this page.