Measuring Errors

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 13 of 58.

How far off is a model, in dollars? Here is a house price model on five test houses it never saw:

Five bars, one per test house, for the size of the price model's miss: 20, 40, 10, 90 and 30 thousand dollars. A dashed line marks the MAE at 38, a dotted one the RMSE at 47.1, above it because of the one big miss of 90

sklearn.metrics measures the misses for you:

from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

y_test = np.array([290, 410, 235, 520, 330])  # thousand dollars
pred = np.array([270, 450, 245, 430, 360])
print(mean_absolute_error(y_test, pred))
print(np.round(np.sqrt(mean_squared_error(y_test, pred)), 1))
print(np.round(r2_score(y_test, pred), 3))

This prints:

38.0
47.1
0.776

The MAE, mean absolute error, is the typical miss: 38 thousand. The RMSE squares the misses, averages, then takes the root: one big miss lifts it to 47.

Both speak the target's units. r2_score is your R squared: 78% of the prices' spread explained.

Challenge

Medium

A shop prices secondhand cameras with a model. Does a second feature help? The program reads shutter_k (thousands of photos shot) or megapixels. The starter fits two models, on age_years alone and with the extra, keeping their test predictions in preds.

For each model, print its MAE, RMSE (1 decimal) and R squared (3 decimals).
Print the one with the smaller MAE.

For another shop and shutter_k:

age_years: MAE 212.4, RMSE 268.1, R squared 0.853
age_years + shutter_k: MAE 98.7, RMSE 121.0, R squared 0.97
Better: age_years + shutter_k, off by 98.7 dollars on a typical camera

Try it yourself

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

# One row per camera sold: age, shutter count in thousands, megapixels, price in dollars
cameras = pd.read_csv("cameras.csv")
extra = input()
names = ["age_years", "age_years + " + extra]
columns = [["age_years"], ["age_years", extra]]
# Split whole rows; each model below picks its own columns
X_train, X_test, y_train, y_test = train_test_split(cameras, cameras["price"], test_size=0.25, random_state=0)
# One model per list of columns, and its predictions for the test cameras
preds = []
for cols in columns:
    model = LinearRegression()
    model.fit(X_train[cols], y_train)
    preds.append(model.predict(X_test[cols]))

# Write your code below