How k-NN Decides

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 6 of 58.

This chapter opens k-NN up. By its end, you will draw its map for any data, like this one for mild and hot chilies:

80 chilies by length and wall thickness on a shaded map: an orange region on the left where the model says hot, a blue region on the right where it says mild

To label a new chili, k-nearest neighbors finds the k training chilies closest to it, and they vote:

A close-up of the chilies with a black star for a new one. Gray lines join it to its 5 nearest chilies: 3 hot and 2 mild, so hot wins

Closest means the smallest distance, the one you met in section 2. You choose k:

model = KNeighborsClassifier(n_neighbors=5)

n_neighbors is k, 5 unless you say otherwise. With two classes, an odd k can never tie.

Challenge

Easy

An astronomy club sorts the stars in stars.csv into dwarfs and giants by temperature and magnitude. The starter reads a new star and k.

Find the k nearest stars by hand: add a distance column, then sort.
Print how many of them are giants, then the vote's winner.
Check with a k-NN model of that k, fitted on every star.

For another club's star and k = 5:

Giants among the nearest 5: 4
By hand: giant
Model: giant

Try it yourself

import numpy as np
import pandas as pd
from sklearn.neighbors import KNeighborsClassifier

# One row per star: temperature in thousand kelvin, magnitude (smaller is brighter), dwarf or giant
stars = pd.read_csv("stars.csv")
X = stars[["temp_kK", "magnitude"]]
y = stars["type"]

# The new star, and k
temp = float(input())
mag = float(input())
k = int(input())
new_star = pd.DataFrame({"temp_kK": [temp], "magnitude": [mag]})

# Write your code below