The Z-Score Rule

Part of the Statistics for ML section of Coddy's Machine Learning journey. Lesson 26 of 64.

A farm weighed 300 eggs, then typed two by hand: 6.2 g and 94.6 g, both typos. Their z-scores (distances from the mean, in std) give them away:

A histogram of 300 egg weights as z-scores, all between -3 and 3 inside a shaded band marked within 3 std. Two red dots sit far outside it: 6.2 g at z = -9.9 and 94.6 g at z = 6.3

np.abs drops minus signs, so one comparison catches both sides:

rng = np.random.default_rng(3)
eggs = np.concatenate([rng.normal(60, 4, size=300), np.array([6.2, 94.6])])
z = (eggs - eggs.mean()) / eggs.std()
print(np.abs(np.array([-4, 2, -1])))
print(eggs[np.abs(z) > 3])

This prints:

[4 2 1]
[ 6.2 94.6]

In bell-shaped data, 99.7% of values lie within 3 std, so np.abs(z) > 3 flags the rare values to check before training.

But an outlier inflates the std it is measured with, so small samples hide it: use the IQR rule there.

Challenge

Medium

A bakery's scale weighed 250 loaves into loaves, and 3 more were typed in by hand. The program reads a cutoff.

Compute every loaf's z-score with the array's .mean() and .std().
In position order, print each loaf whose z-score is more than the cutoff from 0: its position, weight (1 decimal) and z-score (2 decimals).
Print how many you listed.

For another day and the cutoff 3:

Loaf 200: 702.5 g, z = -5.38
Loaf 201: 889.3 g, z = 4.91
Loaves to re-weigh: 2

Try it yourself

import numpy as np

# Don't change the lines below: 250 loaves weighed by the scale, then 3 weights typed in by hand
rng = np.random.default_rng(25)
loaves = np.concatenate([rng.normal(800, 15, size=250), np.array([731.4, 868.9, 912.3])])
cutoff = float(input())

# Write your code below