Apply function to every row in a Pandas DataFrame
Applying a function to every row in a Pandas DataFrame means executing custom logic on each row individually. For example, if a DataFrame contains columns 'A', 'B' and 'C', and you want to compute their sum for each row, you can apply a function across all rows to generate a new column.
Let’s explore efficient methods to achieve this efficiently.
Using vectorized operations
This is the most efficient method for column-wise operations in Pandas. It directly adds entire columns using built-in vectorized arithmetic, leveraging low-level optimizations. Ideal for simple mathematical computations across columns.
import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
df['add'] = df['A'] + df['B'] + df['C']
print(df)
Output
A B C add 0 1 4 7 12 1 2 5 8 15 2 3 6 9 18
Explanation: We compute the row-wise sum of columns 'A', 'B' and 'C' using Pandas' vectorized operations by adding a new column 'add' with df['A'] + df['B'] + df['C'].
Using .values()
By converting selected DataFrame columns to a NumPy array using .values, you can perform fast row-wise computations using NumPy functions like np.sum(). This method is very efficient for numerical data.
import pandas as pd
import numpy as np
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
df['add'] = np.sum(df[['A', 'B', 'C']].values, axis=1)
print(df)
Output
A B C add 0 1 4 7 12 1 2 5 8 15 2 3 6 9 18
Explanation: We compute the row-wise sum of columns 'A', 'B' and 'C' using NumPy’s sum() on the DataFrame’s .values, with axis=1 to sum across columns. The result is stored in a new column 'add'.
Using itertuples()
This method iterates over DataFrame rows as namedtuples, offering better performance than iterrows(). It’s suitable when you need row-wise logic without sacrificing too much speed.
import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
res = []
for row in df.itertuples(index=False):
res.append(row.A + row.B + row.C)
df['add'] = res
print(df)
Output
A B C add 0 1 4 7 12 1 2 5 8 15 2 3 6 9 18
Explanation: We compute the row-wise sum of columns 'A', 'B' and 'C' using itertuples(index=False), summing values for each row and storing the results in a list, which is then assigned to a new column 'add'.
Using list comprehension
A Pythonic and readable way to perform row-wise operations by zipping columns. It’s efficient for small to medium DataFrames where the logic is simple.
import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6], 'C': [7, 8, 9]})
df['add'] = [a + b + c for a, b, c in zip(df['A'], df['B'], df['C'])]
print(df)
Output
A B C add 0 1 4 7 12 1 2 5 8 15 2 3 6 9 18
Explanation: We use zip() to iterate over columns 'A', 'B' and 'C' in parallel, summing their values for each row and storing the result in a new column 'add'.
Related articles
