Finding the Odd Ones

Part of the Unsupervised Learning section of Coddy's Machine Learning journey. Lesson 34 of 51.

This chapter hunts odd rows. By its end, you will circle a server's strange minutes:

A night of web server minutes, requests per second across and latency up: a tight rising cloud, and four lonely minutes circled in red where two detectors agree

An anomaly is a rare row unlike the rest: a fault, a fraud, a typo. Each new one differs, so nobody can label them ahead.

Start with z-scores. StandardScaler turns every column into z-scores, so one line flags rows with any value past 3:

z = StandardScaler().fit_transform(X)
odd = (np.abs(z) > 3).sum(axis=1) > 0

A bridge sensor's strain climbing with temperature. One reading far above the rest is circled red, flagged with strain z past 3. One reading below the trend at 27 degrees is circled orange: missed

But a row can be ordinary in each column and odd only in its mix. Column z-scores miss it.

Challenge

Medium

A supermarket logged a day's averages for its 80 freezer cases. Which need a technician? The program reads a cutoff.

Turn every column of X into z-scores.
For each column, print how many freezers it flags: z-score more than the cutoff from 0.
Print every freezer flagged by any column, in table order.

For another store and 3:

temp_c: 1 flagged
power_kw: 2 flagged
door_openings: 0 flagged
defrost_min: 0 flagged
Freezers to check:
  F05
  F13
  F51

Try it yourself

import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler

# One row per freezer case: a day's averages
freezers = pd.read_csv("freezers.csv")
columns = ["temp_c", "power_kw", "door_openings", "defrost_min"]
X = freezers[columns]
cutoff = float(input())

# Write your code below