Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
pip install numpy==1.26.4 pandas==2.2.3
import numpy, pandas numpy._set_promotion_state("weak_and_warn") x = pandas.DataFrame({"x": [1]}) print(x) Issue Description
If using numpy 1.26, and numpy is set to "weak" or "weak_and_warn" promotion mode (meant to be compatible with the behavior of numpy 2.x), this causes internal pandas functions to fail.
For example, the above command to print a trivial DataFrame results in:
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/benjamin/v/lib/python3.11/site-packages/pandas/core/frame.py", line 1214, in __repr__ return self.to_string(**repr_params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/util/_decorators.py", line 333, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/core/frame.py", line 1394, in to_string return fmt.DataFrameRenderer(formatter).to_string( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/io/formats/format.py", line 962, in to_string string = string_formatter.to_string() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/io/formats/string.py", line 29, in to_string text = self._get_string_representation() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/io/formats/string.py", line 53, in _get_string_representation return self._fit_strcols_to_terminal_width(strcols) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/io/formats/string.py", line 163, in _fit_strcols_to_terminal_width col_lens = Series([Series(ele).str.len().max() for ele in strcols]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/core/series.py", line 584, in __init__ data = sanitize_array(data, index, dtype, copy) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/core/construction.py", line 654, in sanitize_array subarr = maybe_convert_platform(data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/benjamin/v/lib/python3.11/site-packages/pandas/core/dtypes/cast.py", line 138, in maybe_convert_platform arr = lib.maybe_convert_objects(arr) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "lib.pyx", line 2602, in pandas._libs.lib.maybe_convert_objects OverflowError: Python int too large to convert to C long This doesn't happen with numpy 1.26 in its default "legacy" mode. It doesn't happen with numpy 2.x in either "legacy" or "weak" mode.
More information about numpy 1.x versus 2.x and promotion modes is documented here: https://numpy.org/devdocs/numpy_2_0_migration_guide.html#changes-to-numpy-data-type-promotion
Expected Behavior
print(pandas.DataFrame({"x": [1]})) should not crash. It should work properly regardless of the global numpy promotion setting.
Installed Versions
INSTALLED VERSIONS ------------------ commit : 0691c5cf90477d3503834d983f69350f250a6ff7 python : 3.11.2 python-bits : 64 OS : Linux OS-release : 6.1.0-7-amd64 Version : #1 SMP PREEMPT_DYNAMIC Debian 6.1.20-2 (2023-04-08) machine : x86_64 processor : byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8 pandas : 2.2.3 numpy : 1.26.4 pytz : 2024.2 dateutil : 2.9.0.post0 pip : 23.0.1 Cython : None sphinx : None IPython : None adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : None blosc : None bottleneck : None dataframe-api-compat : None fastparquet : None fsspec : None html5lib : None hypothesis : None gcsfs : None jinja2 : None lxml.etree : None matplotlib : None numba : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None psycopg2 : None pymysql : None pyarrow : None pyreadstat : None pytest : None python-calamine : None pyxlsb : None s3fs : None scipy : None sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None xlsxwriter : None zstandard : None tzdata : 2024.2 qtpy : None pyqt5 : None