Silhouette Scores

Part of the Unsupervised Learning section of Coddy's Machine Learning journey. Lesson 8 of 51.

Inertia always improves with k, so it needs an elbow. The silhouette asks each point a fairer question: am I closer to my own group than to the next one?

A black point with solid lines to the 8 points of its own group, mean length a = 1.3, and dashed lines to the 8 points of the nearest other group, mean length b = 3.4. Its silhouette is 0.62

A point scores (b - a) / max(a, b): near 1 snug in its group, near 0 on a border, below 0 in the wrong group. The average scores a clustering:

from sklearn.metrics import silhouette_score
for k in [2, 3, 4]:
    km = KMeans(n_clusters=k, n_init=10, random_state=0)
    km.fit(X)
    print(k, f"{silhouette_score(X, km.labels_):.3f}")

For the apples, this prints:

2 0.534
3 0.657
4 0.545

It peaks at the best k, here 3, where the points feel most at home.

Challenge

Easy

Two forecasters disagree on how many kinds of storm hit their coast. The starter holds 150 storms, peak wind and rain, and reads Ana's k, then Ben's.

Score each k by silhouette, to 3 decimals.
Print whose k fits better.
Print the best k from 2 to 8, with its score.

For another coast:

Ana, k = 2: 0.512
Ben, k = 3: 0.574
Ben's k fits better
Best from 2 to 8: k = 3 (0.574)

Try it yourself

import numpy as np
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

# Don't change the lines below
rng = np.random.default_rng(6)
# Four kinds of storm: typical wind and rain, and how many
kinds = [(30, 8, 40), (12, 34, 35), (21, 21, 30), (11, 9, 45)]
parts = []
for w, r, n in kinds:
    parts.append(np.column_stack([rng.normal(w, 2.5, n), rng.normal(r, 3, n)]))
# 150 storms on one coast, one row each: peak wind (m/s) and rain (mm per hour)
storms = np.round(np.concatenate(parts), 1)

# The two forecasters' numbers of storm kinds
k_ana = int(input())
k_ben = int(input())

# Write your code below