-
- Notifications
You must be signed in to change notification settings - Fork 33.1k
Closed
Labels
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Reproducer
This program starts many threads racing to call sysconfig.get_config_var
:
import sysconfig from multiprocessing.pool import ThreadPool def thread(i): assert sysconfig.get_config_var("srcdir") is not None with ThreadPool() as pool: pool.map(thread, range(100))
When run with Python 3.11.0, this asserts as follows:
Traceback (most recent call last): File "/home/grees/core/junk.py", line 8, in <module> pool.map(thread, range(100)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/grees/github.com/cpython/Lib/multiprocessing/pool.py", line 364, in map return self._map_async(func, iterable, mapstar, chunksize).get() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/grees/github.com/cpython/Lib/multiprocessing/pool.py", line 771, in get raise self._value ^^^^^^^^^^^^^^^^^ File "/home/grees/github.com/cpython/Lib/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) ^^^^^^^^^^^^^^^^^^^ File "/home/grees/github.com/cpython/Lib/multiprocessing/pool.py", line 48, in mapstar return list(map(*args)) ^^^^^^^^^^^^^^^^ File "/home/grees/core/junk.py", line 5, in thread assert sysconfig.get_config_var("srcdir") is not None ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError
Analysis
In the sysconfig
module, get_config_var
calls get_config_vars
which initializes the global variable _CONFIG_VARS
in a thread-unsafe manner.
Use case
The reproducer above is simplified from a multi-threaded build system. Each build thread needs to get some information from sysconfig
, so that the threads are racing to be the first to call get_config_vars
.
Workaround
Adding an initial call to get_config_vars
before starting any thread ensures that _CONFIG_VARS
is reliably initialized.
Metadata
Metadata
Assignees
Labels
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Projects
Status
Done