Scaling for Neighbors

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 9 of 58.

k-NN measures distance, so a feature in big numbers outvotes the rest, as section 2 warned. Here river distance, in meters, drowns out elevation:

Two maps of 160 houses, river distance across and elevation up, flooded or dry. Raw: the map is cut into vertical stripes by distance alone, accuracy 0.6. Scaled: flooded houses get the low corner near the river, accuracy 0.925

Scale first. Fit the scaler on the training rows only, then transform both sets:

scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)
model = KNeighborsClassifier()
model.fit(X_train_s, y_train)
print(model.score(X_test_s, y_test))

This prints 0.925, up from 0.6. The test rows stay unseen: the scaler learns nothing from them.

Challenge

Easy

A smart meter guesses the running appliance from its power (watts) and run time (minutes), in meter.csv. The starter reads k and prints k-NN's accuracy on the raw features.

Fit a StandardScaler on the training rows and transform both sets.
Fit a k-NN model of the same k on the scaled rows; print its accuracy to 3 decimals.
Print the gain.

For another home and k = 5:

Unscaled accuracy: 0.65
Scaled accuracy: 0.875
Scaling gains 0.225

Try it yourself

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler

# One row per appliance run: power in watts, run time in minutes, which appliance
meter = pd.read_csv("meter.csv")
X = meter[["watts", "minutes"]]
y = meter["appliance"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
k = int(input())

# k-NN on the raw features
raw_model = KNeighborsClassifier(n_neighbors=k)
raw_model.fit(X_train, y_train)
raw = raw_model.score(X_test, y_test)
print(f"Unscaled accuracy: {round(raw, 3)}")

# Write your code below