Skip to content

sysconfig.get_config_var is not thread-safe #92452

@gareth-rees

Description

@gareth-rees

Reproducer

This program starts many threads racing to call sysconfig.get_config_var:

import sysconfig from multiprocessing.pool import ThreadPool def thread(i): assert sysconfig.get_config_var("srcdir") is not None with ThreadPool() as pool: pool.map(thread, range(100)) 

When run with Python 3.11.0, this asserts as follows:

Traceback (most recent call last): File "/home/grees/core/junk.py", line 8, in <module> pool.map(thread, range(100)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/grees/github.com/cpython/Lib/multiprocessing/pool.py", line 364, in map return self._map_async(func, iterable, mapstar, chunksize).get() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/grees/github.com/cpython/Lib/multiprocessing/pool.py", line 771, in get raise self._value ^^^^^^^^^^^^^^^^^ File "/home/grees/github.com/cpython/Lib/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) ^^^^^^^^^^^^^^^^^^^ File "/home/grees/github.com/cpython/Lib/multiprocessing/pool.py", line 48, in mapstar return list(map(*args)) ^^^^^^^^^^^^^^^^ File "/home/grees/core/junk.py", line 5, in thread assert sysconfig.get_config_var("srcdir") is not None ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError 

Analysis

In the sysconfig module, get_config_var calls get_config_vars which initializes the global variable _CONFIG_VARS in a thread-unsafe manner.

Use case

The reproducer above is simplified from a multi-threaded build system. Each build thread needs to get some information from sysconfig, so that the threads are racing to be the first to call get_config_vars.

Workaround

Adding an initial call to get_config_vars before starting any thread ensures that _CONFIG_VARS is reliably initialized.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions