Residuals
A residual is how far a point sits above or below the line of best fit. Each one is a small subtraction, and together they tell you whether the line is any good.
Last updated
A line of best fit predicts a y value for every x. Most points do not land exactly on it, and the gap between what a point actually is and what the line says it should be is that point's residual. It is the simplest quantity in regression, a single subtraction, and reading residuals is how you tell a line that fits from a line that only looks like it does.
Drag the points below. Each vertical segment is one residual, blue for points above the line and orange for points below, and the table lists every one of them with the sum of their squares.
| x | y | predicted ŷ | residual y − ŷ |
|---|---|---|---|
| 1 | 4 | 2 | 2 |
| 2 | 2 | 3 | -1 |
| 3 | 2 | 4 | -2 |
| 4 | 5 | 5 | 0 |
| 5 | 6 | 6 | 0 |
| 6 | 8 | 7 | 1 |
| 8 | 9 | 9 | 0 |
Each segment is one residual: blue above the line, orange below. Drag a point and watch the sum of squares change.
Observed minus predicted
For a point with coordinates (x, y), the line predicts a value written ŷ, read "y hat". The residual is
e = y - ŷ
observed minus predicted, always in that order.
Take the five points x = 1, 2, 3, 4, 5 and y = 2, 4, 5, 4, 5. Their line of best fit is y = 0.6x + 2.2. At x = 3 the line predicts 0.6 × 3 + 2.2 = 4, and the actual point is (3, 5), so the residual is 1. At x = 1 it predicts 2.8 and the point is (1, 2), so the residual is -0.8.
| x | observed y | predicted ŷ | residual | residual² |
|---|---|---|---|---|
| 1 | 2 | 2.8 | -0.8 | 0.64 |
| 2 | 4 | 3.4 | 0.6 | 0.36 |
| 3 | 5 | 4 | 1 | 1 |
| 4 | 4 | 4.6 | -0.6 | 0.36 |
| 5 | 5 | 5.2 | -0.2 | 0.04 |
| total | 0 | 2.4 |
What the sign tells you
A positive residual means the point sits above the line: the actual value was higher than predicted, so the line underestimated it. A negative residual means the point sits below the line and the line overestimated. A residual of zero means the point lies exactly on the line.
The size is in the units of y. If y is a test score in points, a residual of -0.8 means the student scored 0.8 points below what the line predicted for their x.
Why they add up to zero
Look at the residual column in the table: -0.8, 0.6, 1, -0.6 and -0.2 add up to exactly 0. That is not luck. It happens for every least squares line with an intercept.
The intercept of that line is chosen as b = ȳ - m x̄. Add up the residuals y - (mx + b) over all n points: the y values add to nȳ, the mx terms add to mnx̄, and the b terms add to nb. Substitute the formula for b and the three parts cancel to zero. In words: the line is placed so the points above it and the points below it balance exactly.
That is also why the residuals have to be squared before they can measure anything. Their plain sum is always 0, for a good line and a bad one alike.
The sum of squared residuals
Square each residual and add them, and you get the sum of squared residuals, often written SSE. For the five points above it is 0.64 + 0.36 + 1 + 0.36 + 0.04 = 2.4.
The line of best fit is defined as the line that makes this number as small as possible. Any other line does worse. The line y = 0.5x + 2.5 also passes through the point of means (3, 4) and looks almost the same on a graph, but its residuals are -1, 0.5, 1, -0.5 and 0, and their squares add to 2.5. The line y = x + 1 scores 4. The minimum, 2.4, belongs to y = 0.6x + 2.2 alone.
Squaring has a side effect worth knowing. A residual of 2 contributes 4 to the sum and a residual of 4 contributes 16, so one point far from the rest pulls the whole line towards itself. Drag a single point to a corner of the plot above and watch the sum jump and the line tilt.
Residual plots
A residual plot puts x on the horizontal axis and each point's residual on the vertical one. The fitted line becomes the horizontal line at zero, and the question becomes simple: is there any pattern left?
A patternless band means the line fits. Residuals scattered above and below zero with no shape, about as wide at every x, say the straight line has captured the trend and what remains is noise.
A curve means a straight line is the wrong shape. Fit a line to the squares 0, 1, 4, 9, 16 and 25 at x = 0 to 5 and you get y = 5x - 10/3 with r ≈ 0.9599, which sounds like an excellent fit. The residuals tell the truth:
They run 10/3, -2/3, -8/3, -8/3, -2/3, 10/3: a U. The line is too high in the middle and too low at the ends every time, which is what fitting a straight line to a curve always looks like. A parabola fits these points exactly, which is what the quadratic regression calculator would find.
A fan means the spread grows. If the residuals are small at one end and large at the other, the line may still be right on average, but its predictions get less precise as x grows. Data that grows by a percentage, such as prices or populations, often looks like this, and an exponential model is the usual next thing to try.
Residuals and outliers
A point with an unusually large residual, far above or below the line compared with the others, is an outlier in y. It deserves a second look: it may be a typing error, or a case that really is different.
A point far to the left or right of the rest is a different kind of problem. Because the line pivots around the point of means, a point far out in x has a long lever and can drag the line towards itself, which can leave it with a small residual even though it changed the line the most. So a small residual does not prove a point is ordinary. In the plot above, drag the rightmost point straight down and watch the line follow it.
Try one
The line predicts 4.6 and the point is at 4, so the point sits 0.6 below the line. If you got 0.6, the subtraction was done the wrong way round: it is always observed minus predicted. For your own data, the linear regression calculator gives the line and its sum of squared residuals, and the correlation coefficient page shows what the strength of the fit means as a single number.
Common questions
- What is a residual?
- The vertical distance from a data point to the fitted line: the observed y minus the y the line predicts at that x. If a point is (3, 5) and the line predicts 4 at x = 3, the residual is 5 minus 4 = 1.
- How do you calculate a residual?
- Put the point's x into the equation of the line to get the predicted value, then subtract that prediction from the point's actual y. Residual = observed minus predicted. The order matters: predicted minus observed gives the same size with the wrong sign.
- What does a negative residual mean?
- The point lies below the line, so the line predicted a value higher than the one observed: it overestimated. A positive residual means the point lies above the line and the line underestimated. A residual of 0 means the point is exactly on the line.
- Why do the residuals add up to zero?
- For a least squares line with an intercept, they always do. The intercept is chosen as b = mean of y minus m times mean of x, which places the line through the point of means, and adding up y minus (mx + b) over all the points then gives n times the mean of y, minus m times n times the mean of x, minus n times b, which is exactly 0.
- What should a good residual plot look like?
- A patternless horizontal band around zero, about as wide at the left as at the right. A curve in the residuals (a U or an upside-down U) means a straight line is the wrong shape for the data. A fan that widens across the plot means the predictions get less reliable as x grows.
- What is the sum of squared residuals?
- Square every residual and add them. It measures how far the points sit from the line overall, and it is the quantity the least squares method makes as small as possible. For x = 1 to 5 and y = 2, 4, 5, 4, 5 the least squares line has a sum of squared residuals of 2.4, and every other straight line scores more.
Want to run this on your own numbers?
Linear Regression Calculator Open the calculator