Skip to content

Race condition when importing collections.abc #125245

Closed
@ngoldbaum

Description

@ngoldbaum

Bug report

Bug description:

Discovered alongside #125243 with similar steps to reproduce. I don't have a simpler way to trigger this than "run the PyO3 tests in a loop" because I think it requires many threads accessing the python runtime simulatenously.

To trigger it, have a rust toolchain and Python installed, clone the PyO3 repo and execute the following command:

while RUST_BACKTRACE=1 UNSAFE_PYO3_BUILD_FREE_THREADED=1 cargo test --lib --features=full -- --test-threads=100; do :; done 

You may also hit some other test failures related to ZoneInfo, see the other issue I opened about that.

You will eventually see a test failure with the following text:

---- exceptions::socket::tests::gaierror stdout ---- thread 'exceptions::socket::tests::gaierror' panicked at src/impl_/exceptions.rs:26:17: failed to import exception socket.gaierror: ImportError: cannot import name 'Mapping' from 'collections.abc' (/Users/goldbaum/.pyenv/versions/3.13.0t/lib/python3.13t/collections/abc.py) 

I slightly modified PyO3 to get a traceback as well (hidden because it's a distractingly long diff):

diff --git a/src/impl_/exceptions.rs b/src/impl_/exceptions.rs index 15b6f53b..de63ad59 100644 --- a/src/impl_/exceptions.rs +++ b/src/impl_/exceptions.rs @@ -1,4 +1,8 @@ -use crate::{sync::GILOnceCell, types::PyType, Bound, Py, Python}; +use crate::{ + sync::GILOnceCell, + types::{PyTracebackMethods, PyType}, + Bound, Py, Python, +}; pub struct ImportedExceptionTypeObject { imported_value: GILOnceCell<Py<PyType>>, @@ -20,8 +24,11 @@ impl ImportedExceptionTypeObject { .import(py, self.module, self.name) .unwrap_or_else(|e| { panic!( - "failed to import exception {}.{}: {}", - self.module, self.name, e + "failed to import exception {}.{}: {}\n{}", + self.module, + self.name, + e, + e.traceback(py).unwrap().format().unwrap(), ) }) } 

And the traceback is:

Traceback (most recent call last): File "/Users/goldbaum/.pyenv/versions/3.13.0t/lib/python3.13t/socket.py", line 55, in <module> import os, sys, io, selectors File "/Users/goldbaum/.pyenv/versions/3.13.0t/lib/python3.13t/selectors.py", line 10, in <module> from collections.abc import Mapping 

So somehow during setup of the socket module, Mapping isn't available yet, but only if many threads are simultaneously touching the Python runtime.

(ping @davidhewitt, we probably want to disable the socket error tests on the free-threaded build as well)

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13bugs and security fixes3.14bugs and security fixestopic-free-threadingtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions