Race on `_PyRuntime.signals.unhandled_keyboard_interrupt` when calling `eval()` concurrently in free-threading mode

[hawkinsp]

Bug report

Bug description:

Run the following code under Python 3.13.1t with thread-sanitizer enabled:

from collections import namedtuple
import concurrent.futures
import functools
import threading

num_threads = 8


def closure(b):
  b.wait()
  namedtuple('mytuple', ['f'])

with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
  for _ in range(100):
    b = threading.Barrier(num_threads)
    for _ in range(num_threads):
      executor.submit(functools.partial(closure, b))

Get:

WARNING: ThreadSanitizer: data race (pid=682761)
  Write of size 4 at 0x56312bb5f168 by thread T6:
    #0 run_eval_code_obj /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1318:53 (python3.13+0x4a02a6) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 run_mod /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1422:19 (python3.13+0x49fa95) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #2 _PyRun_StringFlagsWithName /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1221:15 (python3.13+0x49cfe4) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 PyRun_StringFlags /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1233:12 (python3.13+0x49cfe4)
    #4 builtin_eval_impl /usr/local/google/home/phawkins/p/cpython/Python/bltinmodule.c:1027:18 (python3.13+0x3d8189) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 builtin_eval /usr/local/google/home/phawkins/p/cpython/Python/clinic/bltinmodule.c.h:467:20 (python3.13+0x3d8189)
    #6 cfunction_vectorcall_FASTCALL_KEYWORDS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a010) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #9 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #12 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #15 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #17 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #18 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #21 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #24 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #26 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #27 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #28 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

  Previous write of size 4 at 0x56312bb5f168 by thread T8:
    #0 run_eval_code_obj /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1318:53 (python3.13+0x4a02a6) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 run_mod /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1422:19 (python3.13+0x49fa95) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #2 _PyRun_StringFlagsWithName /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1221:15 (python3.13+0x49cfe4) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 PyRun_StringFlags /usr/local/google/home/phawkins/p/cpython/Python/pythonrun.c:1233:12 (python3.13+0x49cfe4)
    #4 builtin_eval_impl /usr/local/google/home/phawkins/p/cpython/Python/bltinmodule.c:1027:18 (python3.13+0x3d8189) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 builtin_eval /usr/local/google/home/phawkins/p/cpython/Python/clinic/bltinmodule.c.h:467:20 (python3.13+0x3d8189)
    #6 cfunction_vectorcall_FASTCALL_KEYWORDS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a010) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #9 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #12 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #15 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #17 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #18 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #21 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #24 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #26 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #27 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #28 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

I think namedtuple calls eval(), which triggers a race here:

cpython/Python/pythonrun.c

Line 1318 in 0671451

_PyRuntime.signals.unhandled_keyboard_interrupt = 0;

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-128130: Fix unhandled keyboard interrupt data race #129975

[colesbury]

I think there's a race condition here even in the default build where a concurrent eval() may wipe out the unhandled_keyboard_interrupt from the main thread.

[colesbury]

Also seen when running ./python -m test test_exceptions --parallel-threads=10 -v under Tsan in main (3.14). It's in a different function, but still on _PyRuntime.signals.unhandled_keyboard_interrupt.

WARNING: ThreadSanitizer: data race (pid=990837)
  Write of size 4 at 0x555555cd2530 by thread T595:
    #0 _PyErr_Display Python/pythonrun.c:1127 (python+0x43bad3) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #1 thread_excepthook_file Modules/_threadmodule.c:2189 (python+0x52f068) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #2 thread_excepthook Modules/_threadmodule.c:2268 (python+0x52f068)
    #3 cfunction_vectorcall_O Objects/methodobject.c:523 (python+0x1fd56e) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x149161) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #5 PyObject_Vectorcall Objects/call.c:327 (python+0x149161)
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1367 (python+0x8a1ff) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x37d7fc) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #8 _PyEval_Vector Python/ceval.c:1761 (python+0x37d7fc)
    #9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14957b) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x14f138) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #11 method_vectorcall Objects/classobject.c:72 (python+0x14f138)
    #12 _PyVectorcall_Call Objects/call.c:273 (python+0x14b9b7) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #13 _PyObject_Call Objects/call.c:348 (python+0x14be80) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #14 PyObject_Call Objects/call.c:373 (python+0x14be80)
    #15 thread_run Modules/_threadmodule.c:354 (python+0x52e235) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #16 pythread_wrapper Python/thread_pthread.h:242 (python+0x466079) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)

  Previous write of size 4 at 0x555555cd2530 by thread T599:
    #0 _PyErr_Display Python/pythonrun.c:1127 (python+0x43bad3) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #1 thread_excepthook_file Modules/_threadmodule.c:2189 (python+0x52f068) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #2 thread_excepthook Modules/_threadmodule.c:2268 (python+0x52f068)
    #3 cfunction_vectorcall_O Objects/methodobject.c:523 (python+0x1fd56e) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x149161) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #5 PyObject_Vectorcall Objects/call.c:327 (python+0x149161)
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1367 (python+0x8a1ff) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x37d7fc) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #8 _PyEval_Vector Python/ceval.c:1761 (python+0x37d7fc)
    #9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14957b) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x14f138) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #11 method_vectorcall Objects/classobject.c:72 (python+0x14f138)
    #12 _PyVectorcall_Call Objects/call.c:273 (python+0x14b9b7) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #13 _PyObject_Call Objects/call.c:348 (python+0x14be80) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #14 PyObject_Call Objects/call.c:373 (python+0x14be80)
    #15 thread_run Modules/_threadmodule.c:354 (python+0x52e235) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)
    #16 pythread_wrapper Python/thread_pthread.h:242 (python+0x466079) (BuildId: f460519d72c998d19781985effd47fe68db6f3c2)

[nascheme]

It looks like this bug has been fixed by gh-128130 and can be closed.

While testing with TSAN, I found another warning and made a PR to fix it: gh-131126.

[hawkinsp]

Closing, on the assumption the PR linked above fixed this!