Tuning a Pipeline

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 48 of 58.

A pipeline can be tuned like any model, cleaning steps included. Each setting is named by its step, two underscores, then the parameter:

pipe = make_pipeline(SimpleImputer(), DecisionTreeClassifier(random_state=0))
grid = {"simpleimputer__strategy": ["mean", "median", "most_frequent"],
        "decisiontreeclassifier__max_depth": [2, 4, 6, 8]}
search = GridSearchCV(pipe, grid, cv=5)
search.fit(X_train, y_train)

make_pipeline names each step after its class, in lowercase. For 200 model rocket launches, reaching 300 m or not, the 12 settings score:

A grid of mean CV accuracies: rows for the mean, median and most_frequent fills, columns for tree depths 2, 4, 6 and 8. Scores run from 0.733 to 0.820, and the cell for median fills at depth 6 is marked best at 0.820

Every fold refits the whole pipeline, imputer too, so no fold peeks at the rows that test it.

Challenge

Medium

A music school wants to know which violin students will pass their grade exam. The starter reads three values of k, and its grid search over a pipeline stops with an error.

Fix the grid's two keys so the search tunes the imputer's strategy and k-NN's neighbors.
Print the best settings, their cross-validated accuracy and the test accuracy, as below.

Another school, with 2, 4, 6:

Best: mean fills, 6 neighbors
Cross-validated accuracy: 0.720
Test accuracy: 0.820

Try it yourself

import pandas as pd
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline

# Violin students: passed their grade exam (1) or not (0); some practice
# logs and sight-reading scores are missing
students = pd.read_csv("violins.csv")
X = students.drop(columns="passed")
y = students["passed"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
ks = [int(input()), int(input()), int(input())]

# This search stops with an error: fix the grid
pipe = make_pipeline(SimpleImputer(), StandardScaler(), KNeighborsClassifier())
grid = {
    "strategy": ["mean", "median"],
    "n_neighbors": ks,
}
search = GridSearchCV(pipe, grid, cv=5)
search.fit(X_train, y_train)

# Write your code below