Correlation in R: cor(), cor.test() and Correlation Matrices

Correlation Between Two Variables: cor()

Correlation asks: when one variable goes up, does the other tend to go up too (positive), go down (negative), or do its own thing (near zero)? In R that's one call:

print(cor(mtcars$wt, mtcars$mpg))

The answer is about −0.87: heavier cars get worse mileage, and the relationship is strong. That single number is the Pearson correlation coefficient, universally written r.

Reading r

The coefficient always lands between −1 and +1. The sign gives the direction; the magnitude gives the strength:

| |r| | Typical reading | | --- | --- | | 0.0 – 0.2 | negligible | | 0.2 – 0.4 | weak | | 0.4 – 0.6 | moderate | | 0.6 – 0.8 | strong | | 0.8 – 1.0 | very strong |

Treat these bands as conversation-starters, not law - in physics an r of 0.6 is disappointing, in psychology it's a career highlight. Two properties worth internalizing: r has no units (correlating weight-in-tons with mpg gives the same r as weight-in-kilograms with mpg, because r is computed on standardized values), and r only measures linear association - a perfect U-shaped relationship can have r ≈ 0.

And the sentence that has to be said: correlation is not causation. Ice-cream sales and drowning deaths correlate strongly across the months of a year - not because ice cream drowns people, but because summer drives both. A correlation tells you two variables move together; it is silent about why. Maybe x drives y, maybe y drives x, maybe a third thing (the season, in this case) drives both. Deciding among those takes experiments or careful causal reasoning, not a bigger r.

Spearman and Kendall: When Pearson Is the Wrong Tool

Pearson works on the raw values, which makes it sensitive to outliers and blind to curved relationships. The method argument switches to rank-based alternatives:

x <- c(1, 2, 3, 4, 5, 6, 7, 8)
y <- c(1, 4, 9, 16, 25, 36, 49, 300)  # increasing, with a wild last value

print(cor(x, y))                        # Pearson - dragged by the outlier
print(cor(x, y, method = "spearman"))   # ranks: perfectly monotonic
print(cor(x, y, method = "kendall"))

Spearman replaces every value with its rank and then computes Pearson on the ranks. Since y always increases as x increases, every rank lines up and Spearman reports exactly 1 - the outlier's size stops mattering, only its position. Reach for Spearman when the data is ordinal (survey scales), heavily skewed, or when the relationship is monotonic but not straight. Kendall answers a similar question from concordant/discordant pairs; it's more robust in small samples but slower, and Spearman is the common default.

The Correlation Matrix

To scan relationships across many variables at once, hand cor() several numeric columns:

m <- cor(mtcars[, c("mpg", "wt", "hp", "disp")])
print(round(m, 2))

Every variable against every other, with 1s on the diagonal (everything correlates perfectly with itself) and a mirror image across it. Rounding to two decimals matters more than it sounds - the unrounded matrix is a wall of digits, and the point of a matrix is to scan it. Here the scan shows mpg correlates negatively with all three (heavier, more powerful, bigger-engined cars burn more fuel) while wt, hp and disp are all strongly positive with each other - a cluster of "big car" variables that will matter when you get to linear regression and its multicollinearity headaches.

Missing Values: the use Argument

With missing data, cor()'s default is to return NA rather than guess:

d <- mtcars[, c("mpg", "wt", "hp")]
d$wt[c(3, 7)] <- NA  # knock two values out

print(cor(d)[1, 2])
print(cor(d, use = "complete.obs")[1, 2])
print(cor(d, use = "pairwise.complete.obs")[1, 2])

• use = "complete.obs" drops every row containing any NA, then computes the whole matrix from the survivors. Consistent, but wasteful - a missing wt also removes that row from the mpg–hp pair.
• use = "pairwise.complete.obs" computes each cell from all rows where that pair is present. It keeps more data, but different cells rest on different subsets, which can very occasionally produce a matrix that's not internally consistent.

For a couple of stray NAs either is fine; just say which one you used.

Is It Significant? cor.test()

cor() gives a number but no sense of whether it could be noise. cor.test() runs the hypothesis test:

print(cor.test(mtcars$wt, mtcars$mpg))

Walk through the output block by block:

• t = −9.56, df = 30 - the test statistic. The null hypothesis is that the true correlation is zero; the observed r is converted to a t statistic on n − 2 degrees of freedom (32 cars − 2).
• p-value = 1.29e-10 - if the true correlation were zero, the probability of seeing an r this far from zero in a sample of 32 is about 0.0000000001. That's overwhelming evidence the association is real - though remember, the p-value speaks to whether r differs from zero, not to whether the relationship is big or causal.
• 95 percent confidence interval: −0.93 to −0.74 - the plausible range for the true correlation. Often more useful than the p-value: even the optimistic end of this interval is a strong negative correlation.
• sample estimates: cor = −0.87 - the same number cor() gave you.

Small samples deserve extra respect here: with n = 10, correlations of ±0.5 appear by luck alarmingly often, and the wide confidence interval will tell you so. Report the interval, not just the p-value.

Seeing It: Always Plot

A correlation coefficient compresses a whole relationship into one number, and the compression can hide curvature, clusters, or one point doing all the work. Before trusting any r, look at the scatter plot:

plot(mtcars$wt, mtcars$mpg)              # one pair
pairs(mtcars[, c("mpg", "wt", "hp", "disp")])  # every pair in the matrix

pairs() draws a grid of scatter plots matching your correlation matrix - the fastest way to check that the numbers mean what you think. For polished heatmap-style matrix graphics, the corrplot package (install.packages("corrplot"), then corrplot(cor(m))) is the standard tool.

What You Take Away

• cor(x, y) gives Pearson's r: sign is direction, magnitude is strength, always in [−1, 1], no units.
• Correlation measures linear co-movement and says nothing about causation - a lurking third variable is always a candidate.
• method = "spearman" for ranks: ordinal data, outliers, monotonic-but-curved relationships.
• cor(df) on numeric columns gives the matrix; round(, 2) it, and mind the use = argument when data is missing.
• cor.test(x, y) adds the p-value and a confidence interval - report the interval.
• Always look at the scatter plot before believing the number.

Next up: when the question sharpens from "do they move together?" to "is this group's mean different from that one's?" - the t-test.