How Good Is the Fit?

Part of the Statistics for ML section of Coddy's Machine Learning journey. Lesson 39 of 64.

Any cloud gets a line, so fits need grades. A sailor logged wind and boat speeds: compare the line's squared misses with the mean's, the laziest guess:

Two panels of six dots, boat speed against wind. Left: a flat line at the mean with long red gaps to the dots, squared misses 16.23. Right: the fitted line with tiny red gaps, squared misses 0.35

R squared is the share of that spread the line explains:

wind = np.array([6, 8, 10, 12, 15, 18])
boat = np.array([3.1, 4.4, 4.9, 6.2, 6.8, 8.1])
slope, intercept = np.polyfit(wind, boat, 1)
predicted = slope * wind + intercept
ss_res = ((boat - predicted) ** 2).sum()  # the fit: 0.35
ss_tot = ((boat - boat.mean()) ** 2).sum()  # the mean: 16.23
print(np.round(1 - ss_res / ss_tot, 3))

This prints 0.979: the line explains 98% of the speed's spread. 1 is a perfect line, 0 no better than the mean.

It is the standard grade of a model that predicts numbers. For a fitted line, it equals r squared.

Challenge

Medium

A mushroom farm logged 16 growing beds in mushrooms.csv. The program reads a column name, and the starter fits the line that predicts harvest_kg from it. Its report needs r_squared, still empty.

Complete r_squared(y, predicted) so it returns the R squared.
At the end, print the check: the correlation of x and y, squared, times 100, rounded to 1 decimal.

For another farm and the input humidity:

humidity explains 27.0% of the harvest variation
Check with r squared: 27.0%

Try it yourself

import numpy as np
import pandas as pd


# Returns the R squared of the predictions for y
def r_squared(y, predicted):
    # Write your code here
    return 0


beds = pd.read_csv("mushrooms.csv")
column = input()
# Don't change the lines below
x = beds[column]
y = beds["harvest_kg"]
slope, intercept = np.polyfit(x, y, 1)
predicted = slope * x + intercept
r2 = r_squared(y, predicted)
print(f"{column} explains {np.round(r2 * 100, 1)}% of the harvest variation")

# Write your code below