Feature Importance

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 30 of 58.

Which features does a tree lean on? feature_importances_ gives each feature's share of the sorting its questions did, and the shares add up to 1:

Sideways bars for a depth 3 iris tree: petal width 0.58, petal length 0.42, sepal width 0.00, sepal length 0.00

Sorted into a Series, they make a ranked chart. ax.barh draws the bars sideways, so long names fit:

imp = pd.Series(tree.feature_importances_, index=iris.feature_names).sort_values()
ax.barh(imp.index, imp.values)

A 0 means the tree never asked about that feature. The sepals could still matter: the petals already told the tree what they would.

Challenge

Medium

A surf school wants to know what gets beginners standing on the board in their first week. The program reads a depth; the starter trains a tree that deep and sets up the chart.

Put the importances in a Series named by feature, smallest first.
Print the top feature (3 decimals) and the features never asked about, as below.
Draw them as sideways bars and show.

For a swim school:

Top feature: pool_hours (0.612)
Never asked: ['goggles']

Then Coddy describes your chart. Theirs looks like this:

Sideways bars titled What Gets Swimmers Through?, biggest on top: pool_hours about 0.61, can_float 0.21, age 0.10, lessons_per_week 0.08, goggles 0

Try it yourself

import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

depth = int(input())

# One row per beginner: 1 in standing means they stood up on the board within their first week
surf = pd.read_csv("surfschool.csv")
features = ["age", "fitness", "days_surfed", "skated_before", "lesson_hours", "wetsuit_fit"]
X_train, X_test, y_train, y_test = train_test_split(surf[features], surf["standing"], test_size=0.25, random_state=0)
tree = DecisionTreeClassifier(max_depth=depth, random_state=0)
tree.fit(X_train, y_train)

fig, ax = plt.subplots(layout="constrained")  # leaves room for long names
ax.set_title("What Gets Beginners Standing?")
ax.set_xlabel("Importance")

# Write your code below