Scoring a Model

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 4 of 58.

Is a model any good? Its accuracy, the share of test rows it gets right, says. Set it beside a lazy guess:

Two bars of accuracy on 50 test loans: always guessing on time scores 0.78, the k-NN model scores 0.88

78% of this library's test loans came back on time, so answering on time every time scores 0.78. That is the baseline.

from sklearn.metrics import accuracy_score

pred = model.predict(X_test)
print(accuracy_score(y_test, pred))
print(model.score(X_test, y_test))
common = y_train.value_counts().index[0]
print((y_test == common).mean())

This prints 0.88, 0.88, 0.78. score predicts and counts in one call; the baseline always answers the most common training label.

A model earns its place only by beating the baseline.

Challenge

Easy

A beekeeper's sensors watch 160 hives in hives.csv; swarm says whether each hive swarmed. Can one sensor warn her? The starter reads a sensor's name and fits a model on that sensor alone.

Print its test accuracy, rounded to 3 decimals.
Print the baseline, naming the label it always answers.
Print whether the model beats it.

For another apiary and weight_change_kg:

Model on weight_change_kg: 0.775
Always "no": 0.7
Beats the baseline: yes

Try it yourself

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier

# One row per hive: its sensors, and whether it swarmed
hives = pd.read_csv("hives.csv")
sensor = input()
X = hives[[sensor]]
y = hives["swarm"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
model = KNeighborsClassifier()
model.fit(X_train, y_train)

# Write your code below