Skip to content

BUG: _get_common_dtype with date32[pyarrow] and null[pyarrow] #62343

@jbrockmendel

Description

@jbrockmendel

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

left = pd.Series([None], dtype="date32[pyarrow]") right = pd.Series([None], dtype="null[pyarrow]") result = pd.concat([left, right]) >>> result.dtype timestamp[ms][pyarrow]

Issue Description

In ArrowDtype._get_common_dtype we go through

 new_dtype = find_common_type( [ dtype.numpy_dtype if isinstance(dtype, ArrowDtype) else dtype for dtype in dtypes if dtype != null_dtype ] ) 

The date dtype.numpy_dtype gives "M8[ms]", so we get new_dtype="M8[ms]", which gets cast to timestamp[ms][pyarrow].

Expected Behavior

date32[pyarrow]

Installed Versions

Replace this line with the output of pd.show_versions()

Metadata

Metadata

Labels

Arrowpyarrow functionalityBugDtype ConversionsUnexpected or buggy dtype conversionsReshapingConcat, Merge/Join, Stack/Unstack, Explodedatetime.datestdlib datetime.date support

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions