Choosing eps

Part of the Unsupervised Learning section of Coddy's Machine Learning journey. Lesson 22 of 51.

eps is a guess until you look. For the park from Noise Points, measure each point's distance to its 4th nearest, counting itself, like min_samples=4. Sort and plot:

A rising curve of 71 sorted distances: low and flat near 1 m for the picnic points, then a sharp turn at about 3 m, where a dashed line marks eps, and a steep climb for the people alone

Crowd points sit low and flat; lone points shoot up. The knee, where the curve turns up, is a good eps:

from sklearn.neighbors import NearestNeighbors
nn = NearestNeighbors(n_neighbors=4).fit(X)
distances, rows = nn.kneighbors(X)
ax.plot(np.sort(distances[:, -1]))

kneighbors gives each point's distances to its 4 nearest, nearest first, so column -1 is the 4th. np.sort orders them.

Challenge

Medium

Seismologists want DBSCAN to find the fault lines under a year of quakes, with min_samples=5. Choose its eps.

Fit NearestNeighbors(n_neighbors=5) on X and get every quake's distances.
Plot the 5th-nearest distances, sorted.
Run it and read the knee. Keep that value in eps, draw it as a dashed gray line with ax.axhline, and show.

For another region, the knee sat near 4 km:

A chart titled Distance to the 5th Nearest Quake: a curve flat near 2 km for about 145 quakes, then climbing steeply past 50 km, with a dashed gray line at 4 km on the knee

Coddy describes your chart, then runs DBSCAN with your eps.

Try it yourself

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.neighbors import NearestNeighbors

# A year of earthquakes in one region: each epicenter in km
quakes = pd.read_csv("quakes.csv")
X = np.column_stack([quakes["east_km"], quakes["north_km"]])

fig, ax = plt.subplots()
ax.set_title("Distance to the 5th Nearest Quake")
ax.set_xlabel("Quakes, sorted")
ax.set_ylabel("Distance (km)")

# Write your code below