Data Cleaning - More tools
Lesson 7 of 19 in Coddy's Pandas Analytics course.
Remove Duplicate Values
Duplicate rows are exactly the same. If there is a single difference, it will not be considered a duplicate.
df = df.drop_duplicates()Rename Columns
Usually, we would like to have a single convention for column names. For that, we can manually rename the columns:
df = df.rename(columns={"old_name": "new_name"})Change Data Types
df["column name"] = df["column name"].astype(bool)
df["column name"] = df["column name"].astype(int)
Challenge
Easyhe CSV file missing.csv is messy.
Here is the first 5 lines of the file:
ID,COUNTRY,COLOR,SKILL,SKILL_POINTS,UTILIZATION,IS_VALID
1,France,Signal violet,marksmanship,14,0.1924,1
1,France,Signal violet,marksmanship,14,0.1924,1
2,Solomon Islands,Pearl violet,,4,,1
3,Germany,,calligraphy,12,0.88646,0Clean the data:
- Remove duplicate rows.
- Rename all columns to lower-case letters.
- Columns with ones and zeros (Investigate the dataframe to find the right columns.) convert to Boolean columns
To iterate over all columns, you can use the .colunms property:
for column in columns:
# your code hereTo track your progress, print the df: print(df)
Store the final result in the df variable.
Don't print the df to pass the test case!
Try it yourself
# pandas as pd is already imported
df = pd.read_csv("./missing.csv")
# Write your code below
All lessons in Pandas Analytics
4Data Analysis with Pandas
Descriptive StatisticsGrouping and Aggregating DataDifferent AggregationsMerge & Concat2Working with the DataFrame
Understanding DataFramesAccessing DataData Cleaning - Missing dataData Cleaning - More tools3Data Manipulation with Pandas
Return Requested ResultFilter DataAdd & DeleteModify DataModify StringsCustom Modifications