Handling Missing Data
Lesson 11 of 14 in Coddy's Data Manipulation in R course.
Missing values are typically represented as <strong>NA</strong> (Not Available). Let's explore strategies to identify, remove, and impute missing values.
Identifying Missing Data
To check for missing values in R, you can use the following functions:
# Check if a value is NA
is.na(x)
# Count NAs in a vector
sum(is.na(x))
# Identify rows with any NA in a data frame
complete.cases(df)Removing Missing Data
You can remove rows with missing values using these methods:
# Remove rows with any NA
df_clean <- na.omit(df)
# Using dplyr to remove rows with NA in specific columns
library(dplyr)
df_clean <- df %>% filter(!is.na(column_name))Imputing Missing Data
Imputation involves replacing missing values with estimated ones. Here are some common imputation techniques:
# Mean imputation
df$column[is.na(df$column)] <- mean(df$column, na.rm = TRUE)
# Median imputation
df$column[is.na(df$column)] <- median(df$column, na.rm = TRUE)
# Mode imputation for categorical data
mode_value <- names(sort(table(df$category), decreasing = TRUE))[1]
df$category[is.na(df$category)] <- mode_value
Test yourselfThis lesson includes a short quiz. Start the lesson to answer it and track your progress.
Test yourselfThis lesson includes a short quiz. Start the lesson to answer it and track your progress.
Test yourselfThis lesson includes a short quiz. Start the lesson to answer it and track your progress.
Challenge
MediumCreate a function that processes a dataset containing information about patients and their medical test results. The function should perform the following operations:
- Convert the input string into a data frame
- Identify and count the number of missing values in each column
- Remove rows with missing values in the 'age' or 'test_result' columns
- Impute missing values in the 'blood_pressure' column with the median value
- Create a new column 'status' based on the 'test_result':
- If test_result is missing, set status to "Unknown"
- If test_result is less than 50, set status to "Normal"
- If test_result is 50 or greater, set status to "Elevated"
- Print the following information:
- Number of missing values in each column before processing
- Number of rows removed
- Median blood pressure used for imputation
- The processed data frame
Try it yourself
# Read input
con <- file("stdin", "r")
input_string <- suppressWarnings(readLines(con))
# Convert input string to data frame
data <- read.csv(text = input_string, stringsAsFactors = FALSE)
# TODO: Write your code below
# 1. Identify and count missing values
# 2. Remove rows with missing values in 'age' or 'test_result'
# 3. Impute missing values in 'blood_pressure'
# 4. Create 'status' column
# 5. Print required information
# Placeholder for output
cat("Missing values in each column before processing:\n")
print(missing_counts)
cat("\nNumber of rows removed:\n")
print(rows_removed)
cat("\nMedian blood pressure used for imputation:\n")
print(median_bp)
cat("\nProcessed data:\n")
print(data_clean)