Python - Remove duplicate values from a Pandas DataFrame

To remove duplicate values from a Pandas DataFrame, use the drop_duplicates() method. At first, create a DataFrame with 3 columns −

dataFrame = pd.DataFrame({'Car': ['BMW', 'Mercedes', 'Lamborghini', 'BMW', 'Mercedes', 'Porsche'],'Place': ['Delhi', 'Hyderabad', 'Chandigarh', 'Delhi', 'Hyderabad', 'Mumbai'],'UnitsSold': [95, 70, 80, 95, 70, 90]})

Remove duplicate values −

dataFrame = dataFrame.drop_duplicates() 

Example

Following is the complete code −

import pandas as pd # Create DataFrame dataFrame = pd.DataFrame({'Car': ['BMW', 'Mercedes', 'Lamborghini', 'BMW', 'Mercedes', 'Porsche'],'Place': ['Delhi', 'Hyderabad', 'Chandigarh', 'Delhi', 'Hyderabad', 'Mumbai'], 'UnitsSold': [95, 70, 80, 95, 70, 90]}) print"Dataframe...\n", dataFrame # counting frequency of column Car count = dataFrame['Car'].value_counts() print"\nCount in column Car" print(count) # removing duplicates dataFrame = dataFrame.drop_duplicates() print"\nUpdated DataFrame after removing duplicates...\n",dataFrame # counting frequency of column Car after removing duplicates count = dataFrame['Car'].value_counts() print"\nCount in column Car" print(count)

Output

This will produce the following output −

Dataframe...            Car        Place   UnitsSold 0          BMW        Delhi         95 1     Mercedes    Hyderabad         70 2  Lamborghini   Chandigarh         80 3          BMW        Delhi         95 4     Mercedes    Hyderabad         70 5      Porsche       Mumbai         90 Count in column Car BMW            2 Mercedes       2 Porsche        1 Lamborghini    1 Name: Car, dtype: int64 Updated DataFrame after removing duplicates...            Car         Place   UnitsSold 0          BMW        Delhi         95 1     Mercedes    Hyderabad         70 2  Lamborghini   Chandigarh         80 5      Porsche       Mumbai         90 Count in column Car BMW 1 Porsche 1 Lamborghini 1 Mercedes 1 Name: Car, dtype: int64