Closed
Description
Bug report
Bug description:
Sub interpreters have a check (_PyImport_CheckSubinterpIncompatibleExtensionAllowed) for extensions which aren't compatible.
One example of an incompatible extension is readline. It has a global state and shouldn't be imported from a sub interpreter. The issue is that it can be imported dynamically and the crash happens in the module init mechanism, but the check is after (https://github.com/python/cpython/blob/main/Python/importdl.c#L208) meaning it doesn't have the chance to stop the import before the segmentation fault.
I discovered this by writing a simple test harness that goes through the Python test suite and tries to run each test in a sub interpreter. It segfaults on a number of test suites (test_builtin is the first one)
from test.libregrtest.findtests import findtests import _xxsubinterpreters as interpreters # Get a list of tests test_names = findtests() skip_tests = [ # "test_builtin" # Crashes ] def run_test(): import unittest test_cases = unittest.defaultTestLoader.loadTestsFromName(f"test.{test_name}") reasons = "" pass_count = 0 fail_count = 0 for case in test_cases: r = unittest.result.TestResult() case.run(r) if r.wasSuccessful(): pass_count += r.testsRun else: for failedcase, reason in r.failures: reasons += "---------------------------------------------------------------\n" reasons += f"Test case {failedcase} failed:\n" reasons += reason reasons += "\n---------------------------------------------------------------\n" fail_count += 1 for failedcase, reason in r.errors: reasons += ( "---------------------------------------------------------------\n" ) reasons += f"Test case {failedcase} failed with errors:\n" reasons += reason reasons += "\n---------------------------------------------------------------\n" fail_count += 1 interp = interpreters.create() for test in test_names: # Run the test suite if test in skip_tests: print(f"Skipping test {test}") continue print(f"Running test {test}") try: result = interpreters.run_func(interp, run_test, shared={"test_name": test}) except Exception as e: print(f"Test {test} failed with exception {e}") continueThis crashes during the test_builtin suite and any others which dynamically load the readline module.
> ./python.exe -X dev test_in_interp.py .... Running test test_bufio Running test test_builtin Fatal Python error: Segmentation fault Current thread 0x00007ff857472700 (most recent call first): File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 1304 in create_module File "<frozen importlib._bootstrap>", line 813 in module_from_spec File "<frozen importlib._bootstrap>", line 915 in _load_unlocked File "<frozen importlib._bootstrap>", line 1325 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1354 in _find_and_load File "/Users/anthonyshaw/projects/cpython/Lib/pdb.py", line 239 in __init__ File "/Users/anthonyshaw/projects/cpython/Lib/doctest.py", line 386 in __init__ File "/Users/anthonyshaw/projects/cpython/Lib/doctest.py", line 1527 in run File "/Users/anthonyshaw/projects/cpython/Lib/doctest.py", line 2261 in runTest File "/Users/anthonyshaw/projects/cpython/Lib/unittest/case.py", line 589 in _callTestMethod File "/Users/anthonyshaw/projects/cpython/Lib/unittest/case.py", line 636 in run File "/Users/anthonyshaw/projects/cpython/Lib/unittest/case.py", line 692 in __call__ File "/Users/anthonyshaw/projects/cpython/Lib/unittest/suite.py", line 122 in run File "/Users/anthonyshaw/projects/cpython/test_in_interp.py", line 19 in run_test Extension modules: _testcapi, _xxsubinterpreters (total: 2) zsh: segmentation fault ./python.exe -X dev test_in_interp.pyStack trace:
python.exe!Py_TYPE (/Users/anthonyshaw/projects/cpython/Include/object.h:297) python.exe!Py_IS_TYPE (/Users/anthonyshaw/projects/cpython/Include/object.h:330) python.exe!PyObject_TypeCheck (/Users/anthonyshaw/projects/cpython/Include/object.h:487) python.exe!PyModule_GetState (/Users/anthonyshaw/projects/cpython/Objects/moduleobject.c:608) readline.cpython-313td-darwin.so!on_startup_hook (/Users/anthonyshaw/projects/cpython/Modules/readline.c:1029) libedit.3.dylib!rl_initialize (Unknown Source:0) readline.cpython-313td-darwin.so!setup_readline (/Users/anthonyshaw/projects/cpython/Modules/readline.c:1219) readline.cpython-313td-darwin.so!PyInit_readline (/Users/anthonyshaw/projects/cpython/Modules/readline.c:1515) python.exe!_PyImport_LoadDynamicModuleWithSpec (/Users/anthonyshaw/projects/cpython/Python/importdl.c:170) python.exe!_imp_create_dynamic_impl (/Users/anthonyshaw/projects/cpython/Python/import.c:3750) python.exe!_imp_create_dynamic (/Users/anthonyshaw/projects/cpython/Python/clinic/import.c.h:485) python.exe!cfunction_vectorcall_FASTCALL (/Users/anthonyshaw/projects/cpython/Objects/methodobject.c:425) python.exe!_PyVectorcall_Call (/Users/anthonyshaw/projects/cpython/Objects/call.c:273) python.exe!_PyObject_Call (/Users/anthonyshaw/projects/cpython/Objects/call.c:348) python.exe!PyObject_Call (/Users/anthonyshaw/projects/cpython/Objects/call.c:373) python.exe!_PyEval_EvalFrameDefault (/Users/anthonyshaw/projects/cpython/Python/generated_cases.c.h:5382) python.exe!_PyEval_EvalFrame (/Users/anthonyshaw/projects/cpython/Include/internal/pycore_ceval.h:115) python.exe!_PyEval_Vector (/Users/anthonyshaw/projects/cpython/Python/ceval.c:1783) python.exe!_PyFunction_Vectorcall (Unknown Source:0) python.exe!_PyObject_VectorcallTstate (/Users/anthonyshaw/projects/cpython/Include/internal/pycore_call.h:168) CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
Metadata
Metadata
Assignees
Projects
Status
Done