Clusters as Features

Part of the Unsupervised Learning section of Coddy's Machine Learning journey. Lesson 40 of 51.

A straight-line model cannot bend. Phone signal in three villages fades with distance from each village's mast, a shape position alone hides:

135 homes in three villages, east across and north up, each village around a black cross, its k-means center. Dashed lines join one red home to all three centers: 4.9 km, 5.2 km and, nearest, 1.0 km

k-means finds the villages. km.transform gives each row's distance to every center, and .min(axis=1) keeps the nearest:

km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(X_train)
near_train = km.transform(X_train).min(axis=1)
near_test = km.transform(X_test).min(axis=1)
X_train2 = np.column_stack([X_train, near_train])
X_test2 = np.column_stack([X_test, near_test])

With that one extra column, a linear model's test R squared climbs from 0.000 to 0.886.

Fit k-means on the training rows only, like a scaler, then transform both sets.

Challenge

Medium

A festival's sound team wants to know how loud it is where people stand. The starter splits crowd.csv, reads k and prints the score from position alone.

Fit k-means with k clusters on the training positions.
Add each row's distance to its nearest center as a third column, in both sets.
Print the new test R squared and the gain, 3 decimals.

For another festival and k = 4:

Position only: R squared 0.097
With the nearest of 4 centers: R squared 0.734
Gain: 0.637

Try it yourself

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.cluster import KMeans

# 250 festival-goers: where they stood (m) and the sound level there (dB)
crowd = pd.read_csv("crowd.csv")
X = crowd[["x_m", "y_m"]].to_numpy()
y = crowd["sound_db"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
k = int(input())


# Fits a linear regression on the training columns, returns its R squared on the test columns
def test_r2(A_train, A_test):
    model = LinearRegression().fit(A_train, y_train)
    return model.score(A_test, y_test)


before = test_r2(X_train, X_test)
print(f"Position only: R squared {before:.3f}")

# Write your code below