Pandas Drop Column

When working with large datasets, there are often columns that are irrelevant or redundant. Pandas provides an efficient way to remove these unnecessary columns using the `drop()` function. In this article, we will cover various methods to drop columns from a DataFrame.

import pandas as pd

data = {'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]}
df = pd.DataFrame(data)
df = df.drop('B', axis=1)
print(df)

Output:

   A  C
0  1  7
1  2  8
2  3  9

In this example, the DataFrame df will now only contain columns 'A' and 'C'. 

We will load this dataset into a Pandas DataFrame and explore different ways to drop columns. Dataset: data.csv

Method 1: Dropping Columns by Name

The most common method for removing columns is by specifying the column name. drop() function allows us to easily remove one or more columns.

A) Call the drop() method with the column name to drop.

# Drop the 'Gender' column
df = df.drop('Gender', axis=1)
print(df)

Output:

In this example, the Gender column is dropped by specifying its name. The axis=1 argument tells Pandas to drop a column (since axis=0 refers to rows).

B) If we need to drop multiple columns, we can pass a list of column names to the drop() function.

# Drop both 'Age' and 'Gender' columns
df = df.drop(['Age', 'Gender'], axis=1)
print(df)

Output:

This method removes both the Age and Gender columns from the DataFrame. By passing a list of column names, you can drop multiple columns at once.

You can refer this article for more detailed explanation: How to drop one or multiple columns in pandas dataframe

Method 2: Dropping Columns Inplace

If you want to modify the original DataFrame directly, you can use the inplace=True argument. This avoids creating a new DataFrame and applies the change to the existing one.

• Use inplace=True to remove columns directly.

import pandas as pd

data = {'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]}
df = pd.DataFrame(data)

# Drop column 'B' inplace
df.drop('B', axis=1, inplace=True)
print(df)

   A  C
0  1  7
1  2  8
2  3  9

Using inplace=True removes the Salary column from the DataFrame without creating a new variable. The change is applied directly to the original DataFrame.

Method 3: Dropping Columns Using del

Another way to drop a column from a DataFrame is by using the `del` statement. This method directly deletes the column from the DataFrame.

• Use del to remove a column.

import pandas as pd

data = {'A': [10, 20, 30], 'B': [40, 50, 60], 'C': [70, 80, 90]}
df = pd.DataFrame(data)

# Drop column 'C' using del
del df['C']
print(df)

    A   B
0  10  40
1  20  50
2  30  60

The del statement is a simple and direct way to remove a column from the DataFrame. It directly deletes the column, and no new DataFrame is created.

Method 4: Dropping Columns with Missing Values

You can also drop columns that contain missing values using the dropna() method. This is useful when you want to clean the dataset by removing columns with NaN values.

• Use dropna(axis=1) to remove columns with missing values.

import pandas as pd

data = {'A': [10, None, 30], 'B': [None, 50, 60], 'C': [70, 80, 90]}
df = pd.DataFrame(data)

# Drop columns with missing values
df.dropna(axis=1, inplace=True)
print(df)

    C
0  70
1  80
2  90

This removes any column that contains missing values (NaN). You can also specify thresh to drop columns that don’t meet a certain number of non-null values.

You can refer this article for more detailed explanation: Working with Missing Data in Pandas

Recommendation: For general column removal, using drop() is the most straightforward method. For cleaning data with missing values, dropna() is ideal. If you prefer to modify the original DataFrame without creating a new one, use inplace=True.