Random Forests

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 41 of 58.

One decision tree is jumpy: change a few training rows and its questions change. This chapter makes trees steady by asking many of them.

A random forest trains many trees and lets them vote. Here seven trees answer for one geyser:

Seven small trees in a row, each with its vote below: yes, yes, no, yes, yes, no, yes. Arrows lead from every vote to one box that reads Forest: yes, 5 votes to 2

In code, it is one more model to fit:

from sklearn.ensemble import RandomForestClassifier

forest = RandomForestClassifier(n_estimators=100, random_state=0)
forest.fit(X_train, y_train)
print(forest.score(X_test, y_test))

On 300 geyser days this prints 0.893, where one tree scored 0.813. n_estimators is the number of trees.

Many voters, fewer mistakes.

Challenge

Medium

A guide dog school wants to know early which puppies will qualify. The starter splits puppies.csv, scores one tree and reads a number of trees.

Train a forest of that many trees (random_state=0) and print its test accuracy.
Print which model wins and by how much, or A tie.

For another school and 50:

Single tree: 0.740
Forest: 0.860
The forest wins by 0.120

Then try 1: a forest of one tree does surprisingly badly. The next lesson shows why.

Try it yourself

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier

# Puppies at a guide dog school: calmness, focus and startle at loud noises (1 to 10),
# weight, months with their raising family, and qualified as a guide dog (1) or not (0)
puppies = pd.read_csv("puppies.csv")
X = puppies[["calmness", "focus", "noise_startle", "weight_kg", "months_with_family"]]
y = puppies["qualified"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
trees = int(input())

# Don't change the lines below: one tree, to beat
tree = DecisionTreeClassifier(random_state=0)
tree.fit(X_train, y_train)
tree_acc = tree.score(X_test, y_test)
print(f"Single tree: {tree_acc:.3f}")

# Write your code below