gh-101965: Fix usage of Py_EnterRecursiveCall return value in _bisectmodule.c

[OTheDev]

Closes #101965

• Issue: Py_EnterRecursiveCall()/_Py_EnterRecursiveCall() return value misused in a number of places #101965

Automerge-Triggered-By: GH:erlend-aasland

[arhadthedev]

A news entry of gh-102078 covers this PR too so skip-news.

[miss-islington]

@OTheDev: Status check is done, and it's a failure or timed out ❌.

[arhadthedev]

ASAN worker reports use-after-free in drop_gil() previously allocated by alloc_interpreter() and freed by _PyInterpreterState_IDDecref():

Details

==17185==ERROR: AddressSanitizer: heap-use-after-free on address 0x6340005a0860 at pc 0x55e932d96946 bp 0x7f3d1f161e10 sp 0x7f3d1f161e00 READ of size 4 at 0x6340005a0860 thread T8 #0 0x55e932d96945 in drop_gil Python/ceval_gil.c:299 #1 0x55e932dfb311 in _PyThreadState_DeleteCurrent Python/pystate.c:1462 #2 0x55e932e3694f in pythread_wrapper Python/thread_pthread.h:233 #3 0x7f3d2458b608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477 #4 0x7f3d244b0132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132) 0x6340005a0860 is located 96 bytes inside of 113016-byte region [0x6340005a0800,0x6340005bc178) freed by thread T0 here: #0 0x7f3d2480d40f in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:122 #1 0x55e932dfc69d in _PyInterpreterState_IDDecref Python/pystate.c:1017 #2 0x55e932ab54ab in interpid_dealloc Objects/interpreteridobject.c:94 #3 0x55e932d830ba in Py_DECREF Include/object.h:561 #4 0x55e932d830ba in Py_XDECREF Include/object.h:654 #5 0x55e932d830ba in _PyFrame_Clear Python/frame.c:142 #6 0x55e9329027ab in clear_thread_frame Python/ceval.c:1652 #7 0x55e9329027ab in _PyEvalFrameClearAndPop Python/ceval.c:1677 #8 0x55e9329027ab in _PyEval_EvalFrameDefault Python/generated_cases.c.h:771 #9 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #10 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #11 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #12 0x55e932909a5e in do_call_core Python/ceval.c:2626 #13 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #14 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #15 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #16 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #17 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #18 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #19 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #20 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #21 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #22 0x55e932909a5e in do_call_core Python/ceval.c:2626 #23 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #24 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #25 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #26 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #27 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #28 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #29 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #30 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #31 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #32 0x55e932909a5e in do_call_core Python/ceval.c:2626 #33 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #34 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #35 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #36 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #37 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #38 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #39 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #40 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #41 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #42 0x55e932909a5e in do_call_core Python/ceval.c:2626 #43 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #44 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #45 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 previously allocated by thread T0 here: #0 0x7f3d2480da06 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:153 #1 0x55e932df80bd in alloc_interpreter Python/pystate.c:602 #2 0x55e932df80bd in PyInterpreterState_New Python/pystate.c:701 #3 0x55e932df4686 in new_interpreter Python/pylifecycle.c:2001 #4 0x55e932df4686 in _Py_NewInterpreterFromConfig Python/pylifecycle.c:2071 #5 0x7f3d1f4a6f3c in interp_create Modules/_xxsubinterpretersmodule.c:560 #6 0x55e932b32db4 in cfunction_call Objects/methodobject.c:537 #7 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #8 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #9 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #10 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #11 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #12 0x55e932909a5e in do_call_core Python/ceval.c:2626 #13 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #14 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #15 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #16 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #17 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #18 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #19 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #20 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #21 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #22 0x55e932909a5e in do_call_core Python/ceval.c:2626 #23 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #24 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #25 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #26 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #27 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #28 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #29 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #30 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #31 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #32 0x55e932909a5e in do_call_core Python/ceval.c:2626 #33 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #34 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #35 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #36 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #37 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #38 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #39 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #40 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #41 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) Thread T8 created by T0 here: #0 0x7f3d2473a815 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208 #1 0x55e932e36cc2 in PyThread_start_new_thread Python/thread_pthread.h:280 #2 0x55e932f7276b in thread_PyThread_start_new_thread Modules/_threadmodule.c:1177 #3 0x55e932b32d94 in cfunction_call Objects/methodobject.c:548 #4 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #5 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #6 0x55e932d13ce4 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:88 #7 0x55e932d13ce4 in _PyEval_Vector Python/ceval.c:1710 #8 0x55e932d13ce4 in PyEval_EvalCode Python/ceval.c:576 #9 0x55e932e004d2 in run_eval_code_obj Python/pythonrun.c:1702 #10 0x55e932e006e1 in run_mod Python/pythonrun.c:1723 #11 0x55e932e0841d in PyRun_StringFlags Python/pythonrun.c:1592 #12 0x7f3d1f4a878f in _run_script Modules/_xxsubinterpretersmodule.c:459 #13 0x7f3d1f4a878f in _run_script_in_interpreter Modules/_xxsubinterpretersmodule.c:515 #14 0x7f3d1f4a878f in interp_run_string Modules/_xxsubinterpretersmodule.c:741 #15 0x55e932b32db4 in cfunction_call Objects/methodobject.c:537 #16 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #17 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #18 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #19 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #20 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #21 0x55e932909a5e in do_call_core Python/ceval.c:2626 #22 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #23 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #24 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #25 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #26 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #27 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #28 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #29 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #30 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #31 0x55e932909a5e in do_call_core Python/ceval.c:2626 #32 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #33 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #34 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #35 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #36 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #37 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #38 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #39 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #40 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #41 0x55e932909a5e in do_call_core Python/ceval.c:2626 #42 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #43 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #44 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #45 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #46 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #47 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #48 0x55e932a4b608 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #49 0x55e932a4b608 in method_vectorcall Objects/classobject.c:89 #50 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #51 0x55e932909a5e in do_call_core Python/ceval.c:2626 #52 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #53 0x55e932a451e4 in _PyObject_FastCallDictTstate Objects/call.c:143 #54 0x55e932a451e4 in _PyObject_Call_Prepend Objects/call.c:486 #55 0x55e932b9506c in slot_tp_call Objects/typeobject.c:8170 #56 0x55e932a401cf in _PyObject_MakeTpCall Objects/call.c:216 #57 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #58 0x55e932f87ef5 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #59 0x55e932f87ef5 in partial_vectorcall Modules/_functoolsmodule.c:[230](https://github.com/python/cpython/actions/runs/4230678718/jobs/7348285060#step:13:231) #60 0x55e932a42bf7 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #61 0x55e932a42bf7 in PyObject_Vectorcall Objects/call.c:301 #62 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #63 0x55e932a4b789 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #64 0x55e932a4b789 in method_vectorcall Objects/classobject.c:59 #65 0x55e932a43a8d in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f8a8d) #66 0x55e932909a5e in do_call_core Python/ceval.c:2626 #67 0x55e932909a5e in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3707 #68 0x55e932d13ce4 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:88 #69 0x55e932d13ce4 in _PyEval_Vector Python/ceval.c:1710 #70 0x55e932d13ce4 in PyEval_EvalCode Python/ceval.c:576 #71 0x55e932d0c2c1 in builtin_exec_impl Python/bltinmodule.c:1076 #72 0x55e932d0c2c1 in builtin_exec Python/clinic/bltinmodule.c.h:586 #73 0x55e932b33e7b in cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:438 #74 0x55e932a42bf7 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92 #75 0x55e932a42bf7 in PyObject_Vectorcall Objects/call.c:301 #76 0x55e9328faef1 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:3086 #77 0x55e932a4399e in PyObject_Call (/home/runner/work/cpython/cpython/python+0x3f899e) #78 0x55e932e648db in pymain_run_module Modules/main.c:300 #79 0x55e932e65e59 in pymain_run_python Modules/main.c:604 #80 0x55e932e67b28 in Py_RunMain Modules/main.c:689 #81 0x55e932e67b28 in pymain_main Modules/main.c:719 #82 0x55e932e67b28 in Py_BytesMain Modules/main.c:743 #83 0x7f3d[243](https://github.com/python/cpython/actions/runs/4230678718/jobs/7348285060#step:13:244)b5082 in __libc_start_main ../csu/libc-start.c:308 SUMMARY: AddressSanitizer: heap-use-after-free Python/ceval_gil.c:[299](https://github.com/python/cpython/actions/runs/4230678718/jobs/7348285060#step:13:300) in drop_gil Shadow bytes around the buggy address: 0x0c68800ac0b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c68800ac0c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c68800ac0d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c68800ac0e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c68800ac0f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa =>0x0c68800ac100: fd fd fd fd fd fd fd fd fd fd fd fd[fd]fd fd fd 0x0c68800ac110: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c68800ac120: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c68800ac130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c68800ac140: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd 0x0c68800ac150: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Shadow gap: cc ==17185==ABORTING 

[miss-islington]

Status check is done, and it's a success ✅.

[OTheDev]

Thanks @erlend-aasland and @arhadthedev for the help here! Appreciate it!

[erlend-aasland]

Thanks for the PR!

Oleg: I re-ran and ASAN succeeded, so it's probably a flaky test :(

[erlend-aasland]

@iritkatriel, I say we backport this to 3.11 and 3.10.

[iritkatriel]

Agreed.

[miss-islington]

Thanks @OTheDev for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

[miss-islington]

Thanks @OTheDev for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

[miss-islington]

Sorry, @OTheDev, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 0f7a9725300e750947159409abbf5722a4a79f8b 3.11

[miss-islington]

Sorry @OTheDev, I had trouble checking out the 3.10 backport branch.
Please retry by removing and re-adding the "needs backport to 3.10" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 0f7a9725300e750947159409abbf5722a4a79f8b 3.10

[erlend-aasland]

@OTheDev, can you fix the backports? See the devguide if you don't know how to do it.

[OTheDev]

@erlend-aasland Sure, I can work on this in a couple of hours!

[OTheDev]

@erlend-aasland It looks like Py_EnterRecursiveCall() is not in _bisectmodule.c in 3.11 and 3.10, so I think a backport isn't needed?

[erlend-aasland]

Yeah, looks like Dennis added them recently. No backport, then! :)