Choose a Model

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 52 of 58.

Several model families could name the penguins. Which fits best? Give each the same lanes and the same folds, then compare, like these wine models:

Five fold accuracies per candidate on the wine data as dots, with a black bar at each mean: logistic highest and tight at 0.970, k-NN at 0.940, the tree lowest at 0.896 with one fold down at 0.78

The highest mean wins. A small std says it won on every fold, not on one lucky split.

pipe = make_pipeline(prep, model)
scores = cross_val_score(pipe, X_train, y_train, cv=5)

Each fold fits the lanes again on its own training part, so every score stays honest. The test penguins stay locked.

Challenge

Medium

Four candidates want the job. The starter holds your lanes, the names and models lists, and reads a number of folds.

For each candidate, cross-validate the pipeline of prep and that model on the training rows.
Print its mean and std, to 3 decimals.
Print the winner: the highest mean.

For another colony and 5 folds:

k-NN: mean 0.951, std 0.031
Logistic regression: mean 0.964, std 0.012
Random forest: mean 0.958, std 0.027
Decision tree: mean 0.902, std 0.048
Winner: Logistic regression

Try it yourself

import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import make_pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.tree import DecisionTreeClassifier

# The colony from Meet the Penguins, its features and its label
penguins = pd.read_csv("penguins.csv")
numbers = ["bill_length_mm", "bill_depth_mm", "flipper_length_mm", "body_mass_g"]
text = ["island", "sex"]
X = penguins[numbers + text]
y = penguins["species"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, stratify=y, random_state=0)

# From Prepare the Data: the two lanes
number_steps = make_pipeline(SimpleImputer(strategy="median"), StandardScaler())
text_steps = make_pipeline(SimpleImputer(strategy="constant", fill_value="unknown"), OneHotEncoder())
prep = ColumnTransformer([("numbers", number_steps, numbers), ("words", text_steps, text)])

# The candidates
names = ["k-NN", "Logistic regression", "Random forest", "Decision tree"]
models = [KNeighborsClassifier(), LogisticRegression(max_iter=1000),
          RandomForestClassifier(n_estimators=100, random_state=0), DecisionTreeClassifier(random_state=0)]
folds = int(input())

# Write your code below