Skip to content

ENH: Replacing behavior currently provided by pandas.to_numeric using errors="ignore" #59221

Open
@ahobeost

Description

@ahobeost

Feature Type

  • Adding new functionality to pandas

  • Changing existing functionality in pandas

  • Removing existing functionality in pandas

Problem Description

As a User, I am reading in CSV files with mixed types in certain columns.
Specifically, the column can have a value, or a file path to a list of values.
With all current quoting flags, the df.read_csv turns this into an object column where the floats or ints are quoted as well.

Thus, turning all nrs into numerics while keeping all file paths as strings is easily done as follows:
df[column_name] = df[column_name].apply(_pd.to_numeric, errors="ignore")

Thus, as the solution above is being sunsetted, I wish pandas could provide a simple solution to get object columns with mixed dtypes which treat nrs as numerics from csv files.

Feature Description

usage:
pd.to_numeric(preserve_text=True)

implementation

Alternative Solutions

usage:
csv_read(to_numeric_in_mixed_columns=True, preserve_text_in_mixed_columns=True)

Additional Context

#54467
#43280
pypest/pyemu#485

The proposed workaround in the pypest issue is not satisfactory as this should be a common occurrence with a common solution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Closing CandidateMay be closeable, needs more eyeballsEnhancementNeeds DiscussionRequires discussion from core team before further action

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions