Description
Currently, we spill the evaluation stack for most escaping calls, but not all.
For a few calls, notably Py_DECREF
, PyStackRef_CLOSE
and related calls, the stack isn't spilled.
Spilling in those cases would add a fair bit of complexity to the code generator and might produce a small slowdown, but it is worth it.
Knowing that all values are in memory and visible during escaping calls allows us to be much less conservative
when handling reference counts. We can potentially defer many, many more references as the GC is guaranteed to
be able to find all references on the stack. The free-threading build can avoid having to NULL
out the stack
as will only see the valid part of the stack.
There are also benefits unrelated to performance: debuggers and other tools can reliably see values on the stack,
providing more information that just local variables. For example, having the full stack visible during sys.monitoring
INSTRUCTION
events allows reversable debuggers to be implemented more simply.
### Tasks
- [x] Strengthen the code generator to enforce stack discipline on reference lifetimes: references nearer the top of the stack must not outlive those lower down
- [ ] Make `Py_DECREF`, `PyStackRef_CLOSE`, etc. escaping
- [ ] Make `DECREF_INPUTS` support safely closing the input references, ensuring that all other live references (and only live references) are visible on the stack during any escaping calls.
Linked PRs
- GH-128682: Stronger checking of
PyStackRef_CLOSE
andDEAD
. #128683 - GH-128682: Convert explicit loops closing arrays into
DECREF_INPUTS
. #128822 - GH-128682: Change a couple of functions to only steal references on success. #129132
- GH-128682: Make
PyStackRef_CLOSE
escaping. #129404 - GH-128682: Spill the stack pointer in labels, as well as instructions #129618
- GH-128682: Mark two more macros as escaping. #129645
- GH-128682: Account for escapes in
DECREF_INPUTS
#129953