Skip to content

API: Series[EA].fillna fallback behavior with incompatible value #45153

@jbrockmendel

Description

@jbrockmendel
dti = pd.DatetimeIndex([pd.NaT, "2016-01-01"], tz="UTC") tdi = dti - dti[1] pi = dti.tz_localize(None).to_period("D") ci = pi.astype("category") ii = pd.IntervalIndex([None]) pd.Series(dti).fillna("foo") # <- casts pd.Series(tdi).fillna("foo") # <- raises pd.Series(pi).fillna("foo") # <- casts, but untested pd.Series(ci).fillna("foo") # <- raises pd.Series(ii).fillna("foo") # <- raises, untested 

ATM fillna behavior with incompatible values is pretty inconsistent. With numpy dtypes (except dt64 and td64) we always cast. With dt64 and dt64tz we always cast.

With td64 we always raise, but we only have tests for the value being an integer. Back circa 2018 we used to coerce integers to timedelta, so pd.Series(tdi).fillna(1) would interpret the 1 as pd.Timedelta(seconds=1) (not nanos!). We deprecated that coercion and now raise, but I don't think there was much thought given to whether to raise versus cast to object.

With Categorical we raise. We have 2 tests specific to that.

With PeriodDtype we cast and with IntervalDtype we raise, but we have tests for neither.

I don't have a particularly strong opinion on this, but would like to be consistent.

Metadata

Metadata

Assignees

Labels

BugExtensionArrayExtending pandas with custom dtypes or arrays.Missing-datanp.nan, pd.NaT, pd.NA, dropna, isnull, interpolatePDEP missing valuesIssues that would be addressed by the Ice Cream Agreement from the Aug 2023 sprintSeriesSeries data structure

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions