Choosing k

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 7 of 58.

Which k suits the chilies? Try several and score each model twice: on rows it studied, train accuracy, and on hidden ones, test accuracy.

Accuracy by k for the chilies, two lines. Train accuracy starts at 1.0 for k = 1 and falls. Test accuracy starts at 0.8, peaks at 0.92 for k = 13, and both lines drop to about 0.7 beyond k = 80

At k = 1 each chili is its own nearest neighbor, so the model memorizes: perfect on its rows, worse on new ones. A huge k blurs the groups.

ks = range(1, 120, 6)
train_acc = []
test_acc = []
for k in ks:
    model = KNeighborsClassifier(n_neighbors=k)
    model.fit(X_train, y_train)
    train_acc.append(model.score(X_train, y_train))
    test_acc.append(model.score(X_test, y_test))

Pick the k where test accuracy peaks, for now: chapter 7 shows a fairer judge.

Challenge

Easy

A football club logs shots in shots.csv: distance, angle, and goal or miss. Which k predicts best? The starter splits them, lists the ks and labels the chart.

For each k, fit a model and keep its train and test accuracy.
Print the best k by test accuracy, as below.
Draw both lists against ks, labelled train and test, with a legend.

For another club:

Best k: 9 (test accuracy 0.783)

Another club's shots: accuracy by k from 1 to 41. Train accuracy falls from 0.99 at k = 1 to about 0.71. Test accuracy rises from 0.68 to a peak of 0.78 at k = 9, then slides to 0.7

Coddy then describes your chart.

Try it yourself

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier

# One row per shot: distance to goal (m), angle to the goal mouth (degrees), goal or miss
shots = pd.read_csv("shots.csv")
X = shots[["distance_m", "angle_deg"]]
y = shots["result"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
ks = range(1, 42, 4)  # k = 1, 5, 9, ..., 41

fig, ax = plt.subplots()
ax.set_title("Shots: Accuracy by k")
ax.set_xlabel("k (neighbors)")
ax.set_ylabel("Accuracy")

# Write your code below