Grouping and Aggregating Data
Lesson 15 of 19 in Coddy's Pandas Analytics course.
Grouping and aggregating data is critical when dealing with large datasets, as it helps simplify and summarize the data in ways that make it easier to understand and analyze.
In the previous lesson, we calculated statistics about a specific group, but calculating statistics for all groups, might be difficult without additional tools. For this, we have the .groupby() method:
df.groupby('column_name')This will return a GroupBy object that groups the dataframe rows by the values of column_name. You can apply aggregate functions on this GroupBy object to perform computations within those groups.
Once the data is grouped, we can perform several aggregating functions to summarize the data, like:
df.groupby('column_name').sum()
df.groupby('column_name').min()
df.groupby('column_name').max()This will return the sum, min, and max of all other numeric columns in the original dataframe, split by column_name.
Now you can extract a specific column that you are analyzing:
df.groupby('column_name').sum()['other_column']
Challenge
EasyThe CSV file visits.csv contains information about how many visits were made at a particular location at a particular time.
Here is the first 5 lines of the file:
location_id,visits,timestamp
8,1771,27498
9,4187,79919
0,4959,54228
6,7721,41588Create a dictionary with the following keys: min, max, mean, std, sum, and median, and provide a series corresponding to the statistics of column visits for each location id.
Try it yourself
# pandas as pd is already imported
df = pd.read_csv("./visits.csv")
# Write your code below
sum_visits = df.groupby("location_id").sum()["visits"]
res = {
"min": # fill
"max": # fill
"mean": # fill
"std": # fill
"median": # fill
"sum": sum_visits
}
print(res)All lessons in Pandas Analytics
4Data Analysis with Pandas
Descriptive StatisticsGrouping and Aggregating DataDifferent AggregationsMerge & Concat2Working with the DataFrame
Understanding DataFramesAccessing DataData Cleaning - Missing dataData Cleaning - More tools3Data Manipulation with Pandas
Return Requested ResultFilter DataAdd & DeleteModify DataModify StringsCustom Modifications