Too Many Features

Part of the Unsupervised Learning section of Coddy's Machine Learning journey. Lesson 24 of 51.

This chapter squeezes many features into a few. By its end, 13 wine measurements become one map:

178 wines on one map with two new features across and up: three groups, cultivar 1 on the right, cultivar 2 low in the middle, cultivar 3 on the left

Why squeeze? Many features are hard to plot, slow to fit, and often repeat each other, like these two:

Total phenols across and flavanoids up for 178 wines: the dots climb together from the lower left to the upper right, with r = 0.86

Twins carry one idea twice. The correlation matrix finds them, with an r near 1 or -1:

wine = load_wine(as_frame=True).frame
corr = wine.corr()
print(round(corr.loc["flavanoids", "total_phenols"], 2))

This prints 0.86. Keep one of each pair, or merge them: merging is what this chapter builds.

Challenge

Medium

A soap workshop measures 10 things on every batch of soap, and the reports grow long. Find the twins. The program reads a limit; the starter builds corr.

Visit every pair of different columns once.
Print each pair whose r is at least the limit in size, either sign, with r to 2 decimals.
Print how many pairs you found.

For another workshop and limit 0.85:

lye_g and oil_g: 0.91
cure_days and water_left_pct: -0.88
Twin pairs: 2

Try it yourself

import pandas as pd

# A soap workshop: 10 measurements for each of 60 batches of soap
batches = pd.read_csv("soaps.csv")
limit = float(input())
corr = batches.corr()
cols = corr.columns

# Write your code below