Logistic Regression in R: glm() with family = binomial

When the Outcome Is Yes/No

Linear regression predicts a number. But many of the questions worth modeling are binary: does the customer churn, does the patient recover, does the email get clicked. Fitting a straight line to a 0/1 outcome breaks immediately - the line happily predicts probabilities of −0.3 or 1.4, which are nonsense.

Logistic regression fixes this by modeling the probability of the outcome through the log-odds (logit) transform: log(p / (1 − p)) = intercept + slope × x. The log-odds scale runs the whole number line, so a linear equation fits naturally there - and mapping back squeezes every prediction into (0, 1) along the familiar S-shaped curve. The price of the trick: coefficients live on the log-odds scale, and the whole game of reading a logistic regression is translating them back into something humans understand.

Fitting: glm() with family = binomial

glm() (generalized linear model) is lm()'s big sibling; family = binomial selects logistic regression. In mtcars, am records transmission type (1 = manual, 0 = automatic) - do fuel-efficient cars tend to be manual?

fit <- glm(am ~ mpg, data = mtcars, family = binomial)
summary(fit)

Two things before reading the output. First, family = binomial is not optional - omit it and glm() silently fits ordinary least squares. Second, the outcome must be binary: 0/1, logical, or a two-level factor (R models the probability of the second level).

Now the summary, block by block:

• Coefficients - the mpg Estimate is about 0.31, and it is a log-odds slope: each extra mpg adds 0.31 to the log-odds of being manual. Positive means "raises the probability," negative means "lowers" - beyond the sign, nobody's intuition works on this scale, which is why the next section exists.
• z value and Pr(>|z|) - same logic as regression's t-tests (Estimate ÷ Std. Error, then a p-value for "could this be zero?"), just using a normal approximation - hence z instead of t. Here p ≈ 0.011: the association between mpg and transmission type is unlikely to be noise.
• Null deviance vs Residual deviance - deviance is the glm world's badness-of-fit (smaller = better). Null deviance (43.2 on 31 df) is the intercept-only model; residual deviance (29.7 on 30 df) is yours. The drop of about 13.6 for one predictor's worth of df is the glm analogue of "R-squared went up."
• AIC - a model-comparison score balancing fit against complexity; lower wins. Meaningless alone, useful between candidate models on the same data.

From Log-Odds to Odds Ratios: exp(coef())

Exponentiating moves coefficients from the additive log-odds scale to the multiplicative odds scale:

fit <- glm(am ~ mpg, data = mtcars, family = binomial)

print(exp(coef(fit)))
print(exp(confint(fit)))

exp(0.307) ≈ 1.36, and here is the honest sentence pattern to memorize: "each extra mpg multiplies the odds of a manual transmission by about 1.36." An odds ratio above 1 raises the odds, below 1 lowers them, exactly 1 is no effect - which is why the confidence interval verdict for odds ratios is "does the interval exclude 1?" (not zero; zero was the boundary back on the log-odds scale).

Mind the language: odds are not probabilities. Odds = p / (1 − p), so probability 0.75 is odds of 3. Multiplying odds by 1.36 is not the same as multiplying probability by 1.36 - and when the outcome is common, the gap is large. An odds ratio of 2 for a rare outcome behaves like "roughly double the risk"; for an outcome running at 50%, it emphatically does not. Never report an odds ratio with risk-ratio wording ("1.36 times as likely") unless the outcome is rare.

Predicted Probabilities: the type = "response" Gotcha

The single most common logistic-regression bug in the wild:

fit <- glm(am ~ mpg, data = mtcars, family = binomial)
new_cars <- data.frame(mpg = c(15, 20, 25, 30))

print(predict(fit, newdata = new_cars))                      # log-odds!
print(predict(fit, newdata = new_cars, type = "response"))   # probabilities

The first call returns the default type = "link" - predictions on the log-odds scale, negative values and all. The second returns actual probabilities. If your "probabilities" ever come out negative or above 1, this is why. Run the block: a 15-mpg car has essentially no chance of being manual, a 30-mpg car is very likely manual, and the S-curve bends through the middle.

Classification: Thresholding and the Confusion Table

Probabilities become predicted classes by choosing a cutoff - 0.5 being the default choice - and the honest scorecard is a table of predicted vs actual:

fit <- glm(am ~ mpg, data = mtcars, family = binomial)

probs <- predict(fit, type = "response")   # fitted probabilities, one per car
predicted <- probs > 0.5

print(table(actual = mtcars$am, predicted = predicted))

The diagonal cells are correct calls; the two off-diagonal cells are the two different mistakes (predicting manual for an automatic, and the reverse). Overall accuracy alone can flatter a model badly - if 95% of customers don't churn, "predict nobody churns" scores 95% while catching zero churners - so always look at both error types. And 0.5 is a convention, not a law: when the two mistakes have different costs, move the threshold accordingly.

One honesty caveat: this table scores the model on the same data it was fit on, which flatters it. Real evaluation holds out data the model never saw.

Multiple Predictors

Exactly like lm() - add terms with +, and every interpretation gains the qualifier "holding the others constant":

fit2 <- glm(am ~ mpg + qsec, data = mtcars, family = binomial)
summary(fit2)
print(exp(coef(fit2)))

Each exponentiated coefficient is now the odds multiplier for a one-unit increase in that predictor among cars alike on the other predictors. The machinery scales, but so do the caveats from linear regression: correlated predictors reshuffle each other's coefficients.

Cautions

• Complete separation. If a predictor splits the outcome perfectly (every car above some mpg is manual, every one below is automatic), the maximum-likelihood coefficient wants to be infinite. R warns - glm.fit: fitted probabilities numerically 0 or 1 occurred - and reports huge coefficients with absurd standard errors. Don't ship those numbers; simplify the model, get more data, or use a penalized method (the brglm2 or logistf packages).
• Enough events. The binding constraint is the count of the rarer outcome, not total rows. An old rule of thumb wants on the order of 10-15 events per predictor; the 32-car examples here are for teaching the mechanics, not a template for publishable sample sizes.
• Odds ratios are not risk ratios when the outcome is common - covered above, repeated because reviewers will catch it even if you don't.

What You Take Away

• Binary outcome → glm(y ~ x, data = df, family = binomial); never forget the family.
• Raw coefficients are log-odds; exp(coef(fit)) gives odds ratios, and the null value for their intervals is 1.
• The sentence pattern: "each one-unit increase in x multiplies the odds of the outcome by exp(b)."
• predict(..., type = "response") for probabilities - the default returns log-odds, the number-one confusion.
• Classify with a threshold and judge with a confusion table; accuracy alone can lie.
• Watch for separation warnings, count your events, and don't dress odds ratios up as risk ratios.

Next up: the machinery behind every interval you've seen so far - confidence intervals with t.test(), confint() and prop.test().