Recap: Screening Test

Part of the Supervised Learning section of Coddy's Machine Learning journey. Lesson 26 of 58.

Challenge

Medium

Remember the missed-cases cell this chapter opened with? A clinic screens tissue samples for cancer, and a missed malignant sample is the worst mistake. The program reads the recall the clinic needs; the starter loads and scales the samples (1 means malignant) and lists thresholds, strictest first.

Train a logistic regression and print its AUC.
Find the first threshold whose recall reaches the need.
At that threshold, print the counts and precision as below.

For another clinic and 0.9:

AUC: 0.981
Threshold: 0.4
Caught 47 of 51 malignant samples
False alarms: 2
Precision: 0.959

Try it yourself

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score, recall_score, precision_score, confusion_matrix

# The recall the clinic needs, like 0.95
needed = float(input())

# Tissue samples: 1 in y means malignant, the case the screening must catch
cancer = load_breast_cancer()
X = cancer.data
y = 1 - cancer.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# Thresholds to try, strictest first
thresholds = [0.5, 0.4, 0.3, 0.2, 0.1, 0.05]

# Write your code below