Why Forests Work

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 42 of 58.

A forest's trees differ on purpose. Each trains on a bootstrap sample: rows drawn at random with replacement, so some repeat and some sit out.

Each question also picks from a random few features. The trees make different mistakes, and the vote cancels them. Two firefly species:

Two charts of 150 fireflies, flash length across and pause up, blue circles for one species and red triangles for the other. Left, one tree: its border has thin strips reaching out to single fireflies. Right, a forest of 100 trees: one smooth border between the groups

The forest keeps a strip only where most trees agree.

The trees sit in forest.estimators_:

third = forest.estimators_[2]
print(round(third.score(X_test, y_test), 3))

This prints 0.737. The whole forest scores 0.895.

Challenge

Medium

Does a forest beat its own trees? The starter fits forest on riders.csv, riders who did or did not finish a 160 km ride under 6 hours, and sets up the chart.

Score every tree in the forest on the test rows.
Histogram the scores, bins=10, and mark forest_acc with a red vertical line.
Print how many trees the forest beats.

Another ride printed The forest beats 88 of its 100 trees and drew this:

A histogram titled The Forest and Its Trees: tree test accuracies from about 0.53 to 0.85, most near 0.70, and a red vertical line near 0.77, right of most bars

Coddy then describes your chart.

Try it yourself

import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

# Riders of a 160 km charity ride: finished under 6 hours (1) or not (0)
riders = pd.read_csv("riders.csv")
features = ["weekly_km", "long_ride_km", "age", "years_riding", "rest_days", "sleep_hours"]
X = riders[features].to_numpy()  # a plain array, the form the forest's trees expect
y = riders["under_6h"].to_numpy()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)

forest = RandomForestClassifier(n_estimators=100, random_state=0)
forest.fit(X_train, y_train)
forest_acc = forest.score(X_test, y_test)

fig, ax = plt.subplots()
ax.set_title("The Forest and Its Trees")
ax.set_xlabel("Test accuracy")
ax.set_ylabel("Trees")

# Write your code below