Regularization

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 34 of 58.

An overfit linear model leans on big weights to bend toward every noisy row. Ridge regression charges a penalty for big weights, and alpha sets the price.

Here are 20 used guitars, six scaled features predicting the price:

Six lines, one weight per guitar feature, against alpha from 0.01 to 1000. Condition starts at 116, brand at 104, age at minus 83; all shrink toward 0 as alpha grows

It is one swap from LinearRegression, whose weights were [-83. 116. 104. 27. 17. 15.]:

from sklearn.linear_model import Ridge

ridge = Ridge(alpha=100)
ridge.fit(X_train, y_train)
print(np.round(ridge.coef_))

This prints:

[-14.  18.  16.  10.   5.   8.]

Larger alpha, smaller weights, less overfitting. Too large, and the model underfits.

Challenge

Medium

A holiday site predicts cottage prices per night from 8 features, using only 16 cottages. The program reads alpha; the starter splits and scales the data.

Fit a LinearRegression and a Ridge with that alpha.
Print each model's total weight size: its weights without signs, added up and rounded.
Print each test R squared (3 decimals) and the better model, as below.

For another region and 5:

Total weight size: plain 90, ridge 53
Test R squared: plain -0.471, ridge 0.642
Better on new cottages: ridge

Try it yourself

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.metrics import r2_score

alpha = float(input())

# Don't change the lines below: 28 holiday cottages, 8 features and the price per night in euros
cottages = pd.read_csv("cottages.csv")
features = ["size_m2", "bedrooms", "lake_m", "age_years", "town_km", "sauna", "road_noise_db", "forest_m"]
X = cottages[features]
y = cottages["price"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=0)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# Write your code below