Inspection of some cython produced functions causes segfault #26127
Description
Code Sample, a copy-pastable example if possible
from pandas._libs.algos import backfill getattr(backfill, "__kwdefaults__", None)Run with -X faulthandler to get more info.
Problem description
Trying to access __kwdefaults__, either by getattr or hasattr, segfaults the Python process. This occurs at least on macOS when pandas is installed from PyPI using pip install pandas. If it is instead installed via pip install pandas --no-binary :all:, then it does not segfault. There may be other functions which do this, but this is the first we encounter.
This impacts our Python language server as it uses the inspect library to examine libraries without Python source. The code sample above is a minimal repro, but in reality it's being called by inspect.getfullargspec() (which eventually does this). When it segfaults, our process crashes (and on some OSs like macOS produces a visible popup as the OS is tracking these sorts of crashes). See: microsoft/python-language-server#740
cython/cython#1470 looks to be related, and would be fixed in Cython 0.29.6, so maybe a version bump is all that would be required. Given I can do --no-binary to do the compilation locally, it may be that what is building for PyPI is older than what happens on my machine.
Expected Output
Anything, just not crash. Here's what macOS's crash reporter says:
Date/Time: 2019-04-17 16:18:22.448 -0700
OS Version: Mac OS X 10.14.4 (18E226)
Report Version: 12
Bridge OS Version: 3.0 (14Y674)
Anonymous UUID: 5A957B3E-4E8F-3DE2-C606-5B11FE48E6DD
Sleep/Wake UUID: 02FDA72B-8D53-471B-80AE-6514E0B386FB
Time Awake Since Boot: 23000 seconds
Time Since Wake: 2200 seconds
System Integrity Protection: disabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Segmentation fault: 11
Termination Reason: Namespace SIGNAL, Code 0xb
Terminating Process: exc handler [80749]
VM Regions Near 0:
-->
__TEXT 000000010a32a000-000000010a32c000 [ 8K] r-x/rwx SM=COW /usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 algos.cpython-37m-darwin.so 0x0000000116045520 pyx_pf_6pandas_5_libs_5algos_600__defaults + 32
1 algos.cpython-37m-darwin.so 0x0000000116195ed0 __Pyx_CyFunction_get_kwdefaults + 48
2 org.python.python 0x000000010a352587 getset_get + 58
3 org.python.python 0x000000010a379c2f _PyObject_GenericGetAttrWithDict + 180
4 org.python.python 0x000000010a379b20 _PyObject_LookupAttr + 166
5 org.python.python 0x000000010a3d6d66 builtin_getattr + 141
6 org.python.python 0x000000010a34d6f2 _PyMethodDef_RawFastCallKeywords + 495
7 org.python.python 0x000000010a34cc8e _PyCFunction_FastCallKeywords + 44
8 org.python.python 0x000000010a3e1db2 call_function + 636
9 org.python.python 0x000000010a3dac35 _PyEval_EvalFrameDefault + 6594
10 org.python.python 0x000000010a3e26d3 _PyEval_EvalCodeWithName + 1867
11 org.python.python 0x000000010a3d91d0 PyEval_EvalCode + 51
12 org.python.python 0x000000010a40779b run_mod + 54
13 org.python.python 0x000000010a4067c5 PyRun_FileExFlags + 163
14 org.python.python 0x000000010a405e6b PyRun_SimpleFileExFlags + 263
15 org.python.python 0x000000010a41e9b0 pymain_main + 5367
16 org.python.python 0x000000010a41f088 _Py_UnixMain + 56
17 libdyld.dylib 0x00007fff5dead3d5 start + 1
Thread 1:
0 libsystem_kernel.dylib 0x00007fff5dfe586a __psynch_cvwait + 10
1 libsystem_pthread.dylib 0x00007fff5e09e56e _pthread_cond_wait + 722
2 libopenblasp-r0.3.5.dev.dylib 0x000000010af14a3b blas_thread_server + 619
3 libsystem_pthread.dylib 0x00007fff5e09b2eb _pthread_body + 126
4 libsystem_pthread.dylib 0x00007fff5e09e249 _pthread_start + 66
5 libsystem_pthread.dylib 0x00007fff5e09a40d thread_start + 13
Thread 2:
0 libsystem_kernel.dylib 0x00007fff5dfe586a __psynch_cvwait + 10
1 libsystem_pthread.dylib 0x00007fff5e09e56e _pthread_cond_wait + 722
2 libopenblasp-r0.3.5.dev.dylib 0x000000010af14a3b blas_thread_server + 619
3 libsystem_pthread.dylib 0x00007fff5e09b2eb _pthread_body + 126
4 libsystem_pthread.dylib 0x00007fff5e09e249 _pthread_start + 66
5 libsystem_pthread.dylib 0x00007fff5e09a40d thread_start + 13
Thread 3:
0 libsystem_kernel.dylib 0x00007fff5dfe586a __psynch_cvwait + 10
1 libsystem_pthread.dylib 0x00007fff5e09e56e _pthread_cond_wait + 722
2 libopenblasp-r0.3.5.dev.dylib 0x000000010af14a3b blas_thread_server + 619
3 libsystem_pthread.dylib 0x00007fff5e09b2eb _pthread_body + 126
4 libsystem_pthread.dylib 0x00007fff5e09e249 _pthread_start + 66
5 libsystem_pthread.dylib 0x00007fff5e09a40d thread_start + 13
Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x000000010a75fd68 rcx: 0x000000010a75fd50 rdx: 0x00000001177329c0
rdi: 0x000000010a75fd80 rsi: 0x0000000000000000 rbp: 0x00007ffee58d49e0 rsp: 0x00007ffee58d49d0
r8: 0x1678a558da7df69e r9: 0x00007ffee58d4a90 r10: 0x00007fe0bb081a68 r11: 0x00007ffee58d4ae8
r12: 0x0000000116209a08 r13: 0x00000001161a8698 r14: 0x0000000116209a08 r15: 0x0000000000000000
rip: 0x0000000116045520 rfl: 0x0000000000010202 cr2: 0x0000000000000000
Logical CPU: 6
Error Code: 0x00000004
Trap Number: 14
Output of pd.show_versions()
INSTALLED VERSIONS
commit: None
python: 3.7.3.final.0
python-bits: 64
OS: Darwin
OS-release: 18.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
pandas: 0.24.2
pytest: None
pip: 19.0.3
setuptools: 40.8.0
Cython: None
numpy: 1.16.2
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.8.0
pytz: 2019.1
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None