Skip to content

Race on GC enabled state under free threading #129533

New issue
New issue
Closed
Closed
Race on GC enabled state under free threading#129533
Assignees
corona10
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-bugAn unexpected behavior, bug, or error
@hawkinsp

Description

@hawkinsp
hawkinsp
opened on Jan 31, 2025

Bug report

Bug description:

The following code, when run under Python 3.13 commit 0468ea1 built with TSAN, produces a data race error:

import concurrent.futures import functools import gc import threading num_threads = 10 def closure(b): b.wait() for _ in range(100): gc.disable() gc.enable() with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor: for i in range(100): b = threading.Barrier(num_threads) for _ in range(num_threads): executor.submit(functools.partial(closure, b))

TSAN report:

WARNING: ThreadSanitizer: data race (pid=3441855) Write of size 4 at 0x5576617e4e34 by thread T10: #0 PyGC_Disable /usr/local/google/home/phawkins/p/cpython/Python/gc_free_threading.c:1508:22 (python3.13+0x444e39) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #1 gc_disable_impl /usr/local/google/home/phawkins/p/cpython/Modules/gcmodule.c:51:5 (python3.13+0x4d8cbf) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #2 gc_disable /usr/local/google/home/phawkins/p/cpython/Modules/clinic/gcmodule.c.h:45:12 (python3.13+0x4d8cbf) #3 cfunction_vectorcall_NOARGS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:484:24 (python3.13+0x289ed9) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #4 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ead4a) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #5 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1ead4a) #6 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e271b) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #7 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de84a) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #8 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1812:12 (python3.13+0x3de84a) #9 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb3bf) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #10 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x5723a2) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #11 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x5723a2) #12 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb033) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #13 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb033) #14 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb0b5) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #15 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e4902) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #16 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de84a) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #17 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1812:12 (python3.13+0x3de84a) #18 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb3bf) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #19 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef38f) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #20 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef38f) #21 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb033) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #22 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb033) #23 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb0b5) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #24 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564a82) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #25 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bdd87) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) Previous write of size 4 at 0x5576617e4e34 by thread T9: #0 PyGC_Enable /usr/local/google/home/phawkins/p/cpython/Python/gc_free_threading.c:1499:22 (python3.13+0x444dc9) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #1 gc_enable_impl /usr/local/google/home/phawkins/p/cpython/Modules/gcmodule.c:37:5 (python3.13+0x4d8c9f) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #2 gc_enable /usr/local/google/home/phawkins/p/cpython/Modules/clinic/gcmodule.c.h:27:12 (python3.13+0x4d8c9f) #3 cfunction_vectorcall_NOARGS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:484:24 (python3.13+0x289ed9) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #4 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ead4a) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #5 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1ead4a) #6 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e271b) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #7 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de84a) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #8 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1812:12 (python3.13+0x3de84a) #9 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb3bf) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #10 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x5723a2) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #11 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x5723a2) #12 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb033) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #13 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb033) #14 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb0b5) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #15 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e4902) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #16 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de84a) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #17 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1812:12 (python3.13+0x3de84a) #18 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb3bf) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #19 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef38f) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #20 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef38f) #21 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb033) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5) #22 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb033) #23 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb0b5) (BuildId: 64364346a930c8e649018fed7b43c1cc0e8920e5)

This is my attempt to reproduce a race I saw in the JAX tsan CI, which is similar but not identical:

WARNING: ThreadSanitizer: data race (pid=145998) Read of size 4 at 0x55cb2fa81e34 by thread T64 (mutexes: read M0): #0 gc_should_collect /__w/jax/jax/cpython/Python/gc_free_threading.c:1062:59 (python3.13+0x44732b) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80) #1 record_allocation /__w/jax/jax/cpython/Python/gc_free_threading.c:1086:13 (python3.13+0x44732b) #2 gc_alloc /__w/jax/jax/cpython/Python/gc_free_threading.c:1708:5 (python3.13+0x44732b) #3 _PyObject_GC_New /__w/jax/jax/cpython/Python/gc_free_threading.c:1720:20 (python3.13+0x447101) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80) #4 tuple_iter /__w/jax/jax/cpython/Objects/tupleobject.c:1110:10 (python3.13+0x2e0700) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80) #5 PyObject_GetIter /__w/jax/jax/cpython/Objects/abstract.c:2860:25 (python3.13+0x1c1f12) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80) ... Previous write of size 4 at 0x55cb2fa81e34 by thread T70 (mutexes: read M0): #0 PyGC_Disable /__w/jax/jax/cpython/Python/gc_free_threading.c:1508:22 (python3.13+0x444e39) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80) #1 __Pyx_PyType_Ready _traversal.c (_traversal.cpython-313t-x86_64-linux-gnu.so+0x7b16) (BuildId: 0e187ad70a2faa86eeb3f1292897a0491466b409) #2 exec_builtin_or_dynamic /__w/jax/jax/cpython/Python/import.c:815:12 (python3.13+0x4658d0) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80) #3 _imp_exec_dynamic_impl /__w/jax/jax/cpython/Python/import.c:4756:12 (python3.13+0x4658d0) #4 _imp_exec_dynamic /__w/jax/jax/cpython/Python/clinic/import.c.h:513:21 (python3.13+0x4658d0) #5 cfunction_vectorcall_O /__w/jax/jax/cpython/Objects/methodobject.c:512:24 (python3.13+0x28a135) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80) #6 _PyVectorcall_Call /__w/jax/jax/cpython/Objects/call.c:273:16 (python3.13+0x1eb033) (BuildId: c9937216e103905f871b62bf50b66fc5a8e96f80) ...

i.e., it looks like there might be a race between gc_should_collect and PyGC_Disable as well.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

Metadata

Metadata

Assignees

Labels

interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-free-threadingtype-bugAn unexpected behavior, bug, or error

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions