Recap: A/B Test

Part of the Statistics for ML section of Coddy's Machine Learning journey. Lesson 46 of 64.

Challenge

Medium

A shop showed 30 visitors page A or B. Better, or luck? ab.csv has version, seconds and pages; the program reads a column.

Print both means.
Test B's mean minus A's by permutation (pool A first, no other draws) and t-test.
Ship B (yes, up by) only if B is higher and the permutation p below 0.05.
Box plots of A and B: x label Version, y label the column, title as below.

Round to 2 decimals, p to 3. Another shop, seconds:

Mean seconds: A 61.25, B 63.83
Ship version B: no, seconds changed by 2.58 (p = 0.675, t-test 0.672)

Then Coddy describes the chart, which looks like this:

A chart titled A/B Test: seconds, x label Version and y label seconds, with two box plots named A and B at about the same height, medians near 65, and two outlier dots for A at 38 and 86

Try it yourself

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from scipy import stats

visits = pd.read_csv("ab.csv")
column = input()
# Don't change the lines below
rng = np.random.default_rng(3)
shuffles = 1000

# Write your code below