A t-test asks whether a difference is bigger than the noise
Every t-test has the same shape. There is a difference you care about: a sample mean against a claimed value, two group means against each other, or the average change across pairs. There is a standard error: how much that difference would wobble from one sample to the next, estimated from the spread of the data. The t statistic is the first divided by the second, so t = 3 means the difference is three standard errors wide, and the t curve for the right degrees of freedom turns that into a p-value.
The three tests differ only in what the difference and the standard error are. One-sample: the mean minus the hypothesised value, over s divided by root n. Paired: the same thing applied to the list of differences, one per pair, which is why pairing is so powerful when each pair shares a lot of noise. Two-sample: the gap between the means over a standard error built from both spreads. For two samples the page defaults to Welch's version, which does not assume the two groups have the same variance and is what every modern statistics package defaults to; the pooled Student version is a switch away for when a course requires it.
The page keeps what can be kept exact. A mean of 26 is printed as 26 and a variance of 32/3 as 32/3, not 10.67; the standard error is shown as a radical, since it is a square root; the t statistic is the first decimal, because dividing by a root leaves nothing rational to keep. Underneath, every step is listed so the sample summary table can be checked against your own working.
What to know about t-tests
Degrees of freedom are n minus 1 for one sample and for paired data, and n1 plus n2 minus 2 for a pooled two-sample test. Welch's df is a fraction computed from the two variances, and it is often not a whole number.
Pairing removes shared noise. Before-and-after measurements on the same people should be a paired test, never a two-sample one; the two-sample test throws away the pairing and usually finds nothing.
Welch is the safe default. When the two variances really are equal the two versions agree; when they are not, only Welch keeps the promised error rate.
The confidence interval says what the p-value cannot: how big the difference is likely to be. A significant t with an interval from 0.1 to 0.3 is a small effect; the same t with an interval from 2 to 6 is a large one.
Cohen's d is the difference in standard-deviation units, so it can be compared across studies with different scales. Around 0.2 is small, 0.5 medium, 0.8 large.
How to run a t-test
1
Choose the kind of test
One sample against a claimed mean; two independent samples against each other; paired when every value in one list has a partner in the other.
2
Paste the data
Commas, spaces or new lines all work. For a paired test the two lists must be the same length and in the same order.
3
Set the hypothesis and the level
Two-tailed for "different", one-tailed for a direction fixed in advance. Alpha of 0.05 is the convention.
4
Read the summary table first
Check that the means and variances match what you expected. Most wrong t-tests are wrong because a value was typed into the wrong list.
5
Then read t, p and the interval
The verdict says whether the difference cleared the bar; the interval says how big it probably is. Report both.
The ten values have mean 26 and sample variance 32/3. The standard error is the root of 32/30, about 1.033, so t is about 1.94 with 9 degrees of freedom and a two-tailed p of about 0.085. Not significant at 5%: a mean of 26 is within what a sample of ten from a population with mean 24 would produce about one time in twelve.
The first group averages about 84.7, the second about 78.9. Welch's test gives t of about 2.46 on about 9 degrees of freedom and p around 0.036; the pooled version gives the same t on 12 degrees of freedom and p around 0.030. Significant either way, and the interval for the difference runs from roughly 0.5 to 11.2.
Every pair went up by 2 to 4 points. The differences average 19/6, about 3.17, with a small spread, so the paired t is large, about 10.3 on 5 degrees of freedom, and p is about 0.0001. A two-sample test on the same numbers would have found far less, because the spread between people swamps the change within each.
T-test mistakes
Running a two-sample test on paired data. The pairing is the information; throwing it away is how a real change disappears into the noise.
Assuming equal variances without checking. The pooled test is only right when the spreads match; Welch costs almost nothing when they do and saves the test when they do not.
Using the population formula for the variance. A t-test uses the sample variance, dividing by n minus 1, and the page does.
Reading significance as size. A tiny difference is significant with enough data; the interval and Cohen's d say whether it matters.
Choosing the tail after seeing the sign. If the question was "different", the test is two-tailed even when the result went your way.
Testing a heavily skewed metric with a handful of values. With n under about 15 and a long tail, the t curve is a rough guide; a rank test or a bootstrap is the honest alternative.
T-test FAQ
When should I use a t-test instead of a z-test?
Whenever the standard deviation is estimated from the same sample, which is nearly always. The t curve's heavier tails account for that extra uncertainty. A z-test is for the textbook case where the population standard deviation is known in advance, or for very large samples where the two curves coincide.
What is the difference between a paired and an unpaired t-test?
A paired test uses the differences within matched pairs, such as the same person measured twice, so the noise shared by each pair cancels. An unpaired (two-sample) test compares two separate groups. Use paired whenever each value in one list has a natural partner in the other; otherwise two-sample.
Should I use Welch's t-test or Student's?
Welch, unless a course or a journal specifies otherwise. Student's pooled test assumes both groups have the same variance and gives wrong error rates when they do not; Welch drops the assumption and agrees with Student when it holds. This page defaults to Welch and offers pooled as a switch.
How do I read the degrees of freedom for Welch's test?
Welch's degrees of freedom come from a formula on the two sample variances and sizes and are usually not a whole number, such as 9.87. That is normal; the t curve is defined for any positive df. When the two variances and sizes are equal it reduces to n1 plus n2 minus 2.
What does the confidence interval for the difference mean?
The range of values for the true difference that the data is consistent with at the chosen level. If it excludes zero, the test is significant at the matching alpha; its width says how precisely the difference is pinned down. Report it beside the p-value.
What is Cohen's d?
The difference in means divided by the standard deviation, so an effect size in standard-deviation units that can be compared across studies with different scales. Conventionally 0.2 is small, 0.5 medium and 0.8 large, though what counts as meaningful depends on the field.
Can I use this for an A/B test?
For a conversion rate, no: that is a comparison of proportions and the A/B test calculator on the tools pages does it with the z-test for two proportions. For a continuous metric like revenue per user or time on page, yes, if you have the per-user values to paste in; the two-sample Welch test is the right choice.