Correlation

Part of the Statistics for ML section of Coddy's Machine Learning journey. Lesson 30 of 64.

A feature that moves with the target helps a model predict it. The correlation r scores how tightly two columns follow a straight line, from -1 to 1:

Six small scatter plots labelled with their r: -0.9 and -0.5 fall, 0.0 is a shapeless cloud, 0.5 and 0.9 rise, and 1.0 is a perfect rising line

Six weeks of a shop's ads and sales: np.corrcoef returns a 2 by 2 table, each array against each, with r at [0, 1]. A column's .corr gives it directly:

ads = np.array([100, 200, 150, 300, 250, 50])
sales = np.array([430, 520, 560, 760, 610, 410])
print(np.round(np.corrcoef(ads, sales), 2))
shop = pd.DataFrame({"ads": ads, "sales": sales})
print(np.round(shop["ads"].corr(shop["sales"]), 2))

This prints:

[[1.   0.94]
 [0.94 1.  ]]
0.94

At 0.94, weeks with more ads clearly sold more.

Challenge

Medium

A gym tracked 20 members for three months in gym.csv, with the columns visits, classes, minutes_per_visit, age and weight_change_kg. The program reads two column names.

Print the two names and their r, rounded to 2 decimals, as below.
Print Direction: positive when r is above 0.3, negative below -0.3, else none.

For a swimming club's laps and minutes:

laps and minutes: r = 0.95
Direction: positive

Try it yourself

import numpy as np
import pandas as pd

gym = pd.read_csv("gym.csv")
first = input()
second = input()

# Write your code below

Test yourself

This lesson includes a short quiz. Start the lesson to answer it and track your progress.

All lessons in Statistics for ML

3Probability by Simulation

Probability as Frequency The Law of Large Numbers Sums of Dice Expected Value Conditional Probability Recap: Dice Stall Odds

6Relationships

Reading Scatter PlotsCorrelationCorrelation Matrix Correlation Heatmaps Correlation Is Not Cause Recap: Ice Cream and Sun

9Feature Scaling

Why Scale Features Min-Max Scaling Standardization StandardScaler Recap: Climbing Partners

12Final Challenges

Train Delays Air Quality Wine Chemistry Clinic Survey

Practice on your own: Online Python compiler