Grid Search

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 39 of 58.

Tuning by hand means loops inside loops. GridSearchCV cross-validates every combination of settings and keeps the best. For 300 tram trips, late or on time:

A 3 by 3 grid of mean accuracies: tree depth 2, 4, 6 down the side, fewest rows per leaf 1, 5, 20 across. The best cell, depth 4 with leaf 5, scores 0.876 and is framed

A dictionary maps each setting to the values to try. min_samples_leaf is the fewest rows a leaf may hold:

from sklearn.model_selection import GridSearchCV

grid = {"max_depth": [2, 4, 6], "min_samples_leaf": [1, 5, 20]}
search = GridSearchCV(DecisionTreeClassifier(random_state=0), grid, cv=5)
search.fit(X_train, y_train)
print(search.best_params_)
print(f"{search.best_score_:.3f}")
print(f"{search.score(X_test, y_test):.3f}")

This prints:

{'max_depth': 4, 'min_samples_leaf': 5}
0.876
0.813

The search refits the winner on all training rows, so search.score tests it once. Keep grids small: each cell costs 5 fits.

Challenge

Medium

A satellite team labels pixels as cloud or clear sky with k-NN. The program reads the largest k to try; the starter splits the data.

Grid-search every odd k from 1 to that number, with weights "uniform" or "distance" (nearer neighbors count more), cv=5.
Print the best settings, their cross-validated accuracy and the test accuracy, 3 decimals.

For other pixels and 7:

Best k: 7, weights: distance
Cross-validated accuracy: 0.885
Test accuracy: 0.800

Try it yourself

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.neighbors import KNeighborsClassifier

max_k = int(input())

# Don't change the lines below: 360 satellite pixels, 4 light readings each; label 1 means cloud
X, y = make_classification(n_samples=360, n_features=4, n_informative=3, n_redundant=0,
                           flip_y=0.08, class_sep=0.7, random_state=1)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)

# Write your code below