Standardization

Part of the Statistics for ML section of Coddy's Machine Learning journey. Lesson 49 of 64.

Many models learn best when every feature is centered at 0 with a spread of 1. Standardization does that: each value becomes its z-score within its column.

Five pumpkins, weight in kg and width in cm:

Two rows of dots on one ruler in standard deviations: five pumpkin weights from 8 to 40 kg and five widths from 30 to 65 cm, each row centered on its own mean at 0, all between -1.6 and 1.6

Broadcasting standardizes every column at once:

pumpkins = np.array([[30, 60],
                     [8, 30],
                     [22, 55],
                     [40, 65],
                     [15, 40]])
z = (pumpkins - pumpkins.mean(axis=0)) / pumpkins.std(axis=0)
print(np.round(z.mean(axis=0), 2))
print(np.round(z.std(axis=0), 2))

This prints:

[0. 0.]
[1. 1.]

Round before you check: unrounded, a mean of 0 can print as 2.22044605e-17, a speck of rounding error.

Unlike min-max, there is no fixed range: most values land between -2 and 2.

Challenge

Medium

A dairy farm spots unusual cows. Each row of cows is a cow: milk (liters a day), weight (kg) and age (years), named in features. The program reads a cow's position k.

Complete standardize(X). The check lines then print [0. 0. 0.] and [1. 1. 1.].
Find cow k's z-score farthest from 0, sign ignored, and print it as below, to 2 decimals.

For a cow on another farm:

Column means: [0. 0. 0.]
Column stds: [1. 1. 1.]
Cow 2 stands out by weight: 1.76 std above the herd

Try it yourself

import numpy as np

features = ["milk", "weight", "age"]
# One row per cow: milk in liters per day, weight in kg, age in years
cows = np.array([[28, 610, 6],
                 [22, 540, 3],
                 [35, 680, 6],
                 [31, 590, 9],
                 [19, 650, 4],
                 [26, 520, 2],
                 [30, 600, 7]])
k = int(input())


# Returns X with every column standardized: mean 0 and std 1
def standardize(X):
    # Replace the line below with the standardization
    return X


# Don't change the three lines below: they check your function
z = standardize(cows)
print("Column means:", np.round(z.mean(axis=0), 2))
print("Column stds:", np.round(z.std(axis=0), 2))

# Write your code below: what stands out about cow k

Test yourself

This lesson includes a short quiz. Start the lesson to answer it and track your progress.

All lessons in Statistics for ML

3Probability by Simulation

Probability as Frequency The Law of Large Numbers Sums of Dice Expected Value Conditional Probability Recap: Dice Stall Odds

6Relationships

Reading Scatter Plots Correlation Correlation Matrix Correlation Heatmaps Correlation Is Not Cause Recap: Ice Cream and Sun

9Feature Scaling

Why Scale Features Min-Max ScalingStandardizationStandardScaler Recap: Climbing Partners

12Final Challenges

Train Delays Air Quality Wine Chemistry Clinic Survey

Practice on your own: Online Python compiler