Grouping and Aggregating with Pandas

When working with large datasets it's used to group and summarize the data to make analysis easier. Pandas a popular Python library provides powerful tools for this. In this article you'll learn how to use Pandas' groupby() and aggregation functions step by step with clear explanations and practical examples. 

Aggregation in Pandas

Aggregation means applying a mathematical function to summarize data. It can be used to get a summary of columns in our dataset like getting sum, minimum, maximum etc. from a particular column of our dataset. The function used for aggregation is agg() the parameter is the function we want to perform. Some functions used in the aggregation are:

• sum()         : Compute sum of column values
• min()          : Compute min of column values
• max()         : Compute max of column values
• mean()       : Compute mean of column
• size()          : Compute column sizes
• describe()  : Generates descriptive statistics
• first()          : Compute first of group values
• last()          : Compute last of group values
• count()       : Compute count of column values
• std()           : Standard deviation of column
• var()           : Compute variance of column
• sem()         : Standard error of the mean of column

Creating a Sample Dataset

Let's create a small dataset of student marks in Maths, English, Science and History.

import pandas as pd

df = pd.DataFrame([[9, 4, 8, 9],
                   [8, 10, 7, 6],
                   [7, 6, 8, 5]],
                  columns=['Maths',  'English', 
                           'Science', 'History'])

print(df)

Output:

Now that we have a dataset let’s perform aggregation.

1. Summing Up All Values (sum())

The sum() function adds up all values in each column.

df.sum()

Output:

2. Getting a Summary (describe())

Instead of calculating sum, mean, min and max separately we can use describe() which provides all important statistics in one go.

df.describe()

Output:

3. Applying Multiple Aggregations at Once (agg())

The .agg() function lets you apply multiple aggregation functions at the same time.

df.agg(['sum', 'min', 'max'])

Output:

Grouping in Pandas

Grouping in Pandas means organizing your data into groups based on some columns. Once grouped you can perform actions like finding the total, average, count or even pick the first row from each group. This method follows a split-apply-combine process:

• Split the data into groups
• Apply some calculation like sum, average etc.
• Combine the results into a new table.

Let’s understand grouping in Pandas using a small bakery order dataset as an example.

import pandas as pd

data = {
    'Item': ['Cake', 'Cake', 'Bread', 'Pastry', 'Cake'],
    'Flavor': ['Chocolate', 'Vanilla', 'Whole Wheat', 'Strawberry', 'Chocolate'],
    'Price': [250, 220, 80, 120, 250]
}

df = pd.DataFrame(data)
print(df)

Output:

Grouping Data by One Column Using groupby()

Let’s say we want to group the orders based on the Item column.

grouped = df.groupby('Item')
print(grouped)

Output:

<pandas.core.groupby.generic.DataFrameGroupBy object at 0x7867484be150>

This doesn't show the result directly it just creates a grouped object. To actually see the data we need to apply a method like .sum(), .mean() or first(). Let’s find the total price of each item sold:

print(df.groupby('Item')['Price'].sum())

Output:

The above output shows the total earnings from each item.

Grouping by Multiple Columns

Now let’s group by Item and Flavor to see how each flavored item sold.

print(df.groupby(['Item', 'Flavor'])['Price'].sum())

Output:

The above output show Chocolate Cakes earned ₹500 and Vanilla Cake earned ₹220 and more.