Seeing 64 Dimensions

Part of the Unsupervised Learning section of Coddy's Machine Learning journey. Lesson 29 of 51.

This chapter puts PCA to work on handwritten digits. By its end, you squeeze a scan into a few numbers and rebuild it:

One handwritten 9 rebuilt five times: from 2 numbers a gray blur, from 5 and 10 a rough 9, from 20 a clear 9, and the real 64 pixel scan last

First, look. A scan is 64 pixels, and no chart has 64 axes, so PCA squeezes them to 2:

digits = load_digits()
X = digits.data
y = digits.target
points = PCA(n_components=2).fit_transform(X)
for d in range(10):
    ax.scatter(points[y == d, 0], points[y == d, 1], label=str(d))

No scaler here: every pixel shares one unit, ink from 0 to 16.

Digit scans as dots on two components, each digit labelled at its center: the 0s at the bottom, the 4s far right, the 3s far left, the other digits in patches between them, some overlapping

PCA never saw y; it only colors the dots. Yet each digit gathers in its own patch: the structure was in the pixels.

Challenge

Medium

A handwriting app for kids wants to know which digits children could mix up. The program reads some digits, typed together like 358, into chosen.

Squeeze all 1,797 scans to 2 numbers each with PCA, into points.
Scatter each chosen digit's scans, labelled with the digit. Add a legend and show.
Call report_closest(points, y, chosen).

For 046, it prints:

Closest on the map: 4 and 6 (13.6 apart)

and draws this. Then Coddy describes your chart:

A scatter titled Digits that look alike with a legend: the 0s low on the left, the 6s low in the middle, the 4s high on the right, the 6s brushing against the 4s

Try it yourself

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA


# From Measuring Similarity: prints the two chosen digits whose centers lie closest on the map
def report_closest(points, y, chosen):
    best = -1
    for i in range(len(chosen)):
        for j in range(i + 1, len(chosen)):
            a = points[y == chosen[i]].mean(axis=0)
            b = points[y == chosen[j]].mean(axis=0)
            dist = np.sqrt(((a - b) ** 2).sum())
            if best == -1 or dist < best:
                best = dist
                pair = (chosen[i], chosen[j])
    print(f"Closest on the map: {pair[0]} and {pair[1]} ({best:.1f} apart)")


# 1,797 scans: 64 pixels each in X, the digit drawn in y
digits = load_digits()
X = digits.data
y = digits.target

# The digits to compare, typed together like 358
chosen = []
for ch in input():
    chosen.append(int(ch))

# Don't change the lines below
fig, ax = plt.subplots()
ax.set_title("Digits that look alike")
ax.set_xlabel("Component 1")
ax.set_ylabel("Component 2")

# Write your code below