Look at the Digits

Part of the Unsupervised Learning section of Coddy's Machine Learning journey. Lesson 43 of 51.

Your last project: 1,797 handwritten scans and no teacher. Over five lessons, you will find groups like these without reading a single label:

Ten blurry gray digit pictures in two rows, the centers of ten groups k-means found in the scans: a 9, a blurred 1 or 8, a 7, a 6, a 2, a 0, a 3, a 4, a 5 and a thin 1

Every project starts by looking. A pixel's spread, its std over many scans, shows where writers disagree. Here, over every 7:

An 8 by 8 grid of the sevens' pixel spreads, darker blue for more: high along the top bar and the stroke down, 0.0 down the left edge and at the bottom right

spread = X[y == 7].std(axis=0)
print((spread == 0).sum())

This prints 15: fifteen pixels are blank in every 7. A pixel that never changes tells a model nothing.

Challenge

Medium

Start exploring. The starter loads the scans into X, the digits into y, and sets up a chart.

Find each pixel's spread over all the scans. Print the busiest pixel's row, column and spread (2 decimals), then how many pixels stay blank in every scan.
Draw them as an 8 by 8 picture on ax, cmap="Blues", with a color bar. Show it.

For the 1s alone:

Busiest pixel: row 6, column 5 (spread 6.53)
Blank in every scan: 12 pixels

An 8 by 8 picture titled Where Writers Disagree with a color bar: the spread of each pixel over the 1s, rows and columns numbered 0 to 7: two dark bands down the middle, where writers put the stroke in different places, and pale edges left and right

Yours covers all 1,797 scans. Coddy describes your picture. Where do writers disagree most?

Try it yourself

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits

# 1,797 handwritten scans: 64 pixels each (8 by 8, ink from 0 to 16) and the digit written
digits = load_digits()
X = digits.data
y = digits.target

fig, ax = plt.subplots()
ax.set_title("Where Writers Disagree")

# Write your code below