Dendrograms

Part of the Unsupervised Learning section of Coddy's Machine Learning journey. Lesson 18 of 51.

The chapter's opening tree is a dendrogram. Each bracket joins two groups at the height where they merged: the taller its legs, the farther apart they were.

Cut it with a line, and every branch the line crosses becomes a group. Here, 4:

The sports ball tree with a dashed line at height 1.2. Below it, four groups: softball, cricket and baseball; tennis, table tennis and golf; basketball alone; volleyball, handball and football

scipy builds the tree, draws it, and cuts it with fcluster:

from scipy.cluster.hierarchy import linkage, dendrogram, fcluster
Z = linkage(scaled, "ward")
dendrogram(Z, labels=names, leaf_rotation=90, color_threshold=1.2, ax=ax)
ax.axhline(1.2, color="gray", linestyle="--")
groups = fcluster(Z, t=1.2, criterion="distance")
print(groups)

This prints a group number per ball, the basketball alone:

[2 2 2 1 1 1 3 3 3 4]

Cut across the longest legs: the groups below are far apart.

Challenge

Medium

A marine guide groups 16 whales and dolphins by length, weight and diving depth. The starter scales them and reads the cut height, cut.

Build the ward tree Z and draw it on ax: animal names turned 90 degrees, colors stopping at the cut.
Draw the cut as a gray dashed line, then show.
Print the number of groups.

For 11 fruits and a cut at 3.5:

Cut at 3.5: 3 groups

A tree titled Fruits, leaf names running upward, and a gray dashed line at height 3.5 crossing three colored branches: lemon, lime and strawberry; orange, grapefruit and mango; grape, cherry, banana, apple and pear

Coddy describes your tree.

Try it yourself

import pandas as pd
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from scipy.cluster.hierarchy import linkage, dendrogram, fcluster

# 16 whales and dolphins: body length (m), weight (tonnes), deepest dive (m)
whales = pd.read_csv("whales.csv")
names = list(whales["animal"])
scaled = StandardScaler().fit_transform(whales[["length_m", "weight_t", "dive_m"]])
cut = float(input())

fig, ax = plt.subplots()
fig.subplots_adjust(bottom=0.3)  # room for the names below the tree
ax.set_title("Whales and Dolphins")
ax.set_ylabel("Merge height")

# Write your code below