Formulae for Correlation and Regression Revision Notes for HSC SSCE Mathematics Advanced

Formulae for Correlation and Regression

Introduction

While technology is commonly used to calculate Pearson's correlation coefficient and the line of best fit, understanding the underlying formulae is essential in mathematics. This note presents the actual formulae for correlation and regression, helping you understand what happens "behind the scenes" when using technology.

infoNote

The formulae are quite detailed and calculations can be time-consuming compared to finding means and standard deviations. However, working through examples by hand develops a deeper understanding of these statistical measures and helps you verify results from technology.

The formula for Pearson's correlation coefficient

Pearson's correlation coefficient $r$ is the standard measure of correlation. It tests specifically for linear correlation, meaning it measures how close data points are to lying on a straight line with a non-zero gradient.

The formula is:

$r = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sqrt{\sum (x - \bar{x})^2 \sum (y - \bar{y})^2}}$

Understanding the formula

Let's break down each component to understand what this formula is measuring:

Step 1: Calculate the means

First, calculate $\bar{x}$ and $\bar{y}$ , which are the means of the $x$ -values and $y$ -values respectively. The point $(\bar{x}, \bar{y})$ will lie in the middle of the data cluster on the scatterplot.

Step 2: Find deviations from the mean

Calculate all the deviations from the mean:

$(x - \bar{x})$ represents how far each $x$ -value is from the mean $\bar{x}$
$(y - \bar{y})$ represents how far each $y$ -value is from the mean $\bar{y}$

Step 3: The numerator - products of deviations

The numerator involves the product $(x - \bar{x})(y - \bar{y})$ for each data point. This is the key component because:

If $x$ and $y$ both lie on the same side of their respective means, the product is positive
If they lie on opposite sides, the product is negative

When we sum these products, we get a sense of whether the variables are working together (positive correlation) or working against each other (negative correlation).

Step 4: The denominator - normalisation

The denominator normalises the quantity and makes it a ratio. Importantly:

The units of $x$ cancel out
The units of $y$ cancel out
The result $r$ is a pure number with no units

infoNote

The denominator is closely related to the standard deviation formulae for $x$ and $y$ . In fact, the formula for $r$ can be rewritten using standard deviations, which provides an alternative computational approach.

Properties of Pearson's correlation coefficient

chatImportant

Critical Properties to Remember:

$r$ is unaffected by units or changes in scale
Changing metres to centimetres doesn't change $r$
Multiplying all $y$ -values by a constant (like $+7$ ) doesn't change $r$ (except possibly the sign)
Only the clustering of data points and the sign of the gradient affect the value of $r$

Formulae for the regression line

The least squares regression line is found by minimising the sum of the squares of the vertical distances from each data point to the line.

The regression line equation is:

$y = mx + b$

where the gradient is:

$m = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sum (x - \bar{x})^2}$

and the $y$ -intercept is:

$b = \bar{y} - m\bar{x}$

infoNote

Key observations about the regression formulae:

Connection to correlation:

The numerator of the gradient $m$ is the same as the numerator of $r$
The denominator of $m$ has already been calculated when finding $r$
Once you've calculated $r$ , finding the regression line is quick!

The $y$ -intercept:

The value of $b$ ensures that the regression line passes through the point $(\bar{x}, \bar{y})$
This is a useful check: the line of best fit always goes through the mean point

Worked example: Calculating correlation and regression

Let's work through a complete example using data about caller waiting times.

lightbulbExample

Worked Example: Calculating Correlation and Regression Line

Step 1: Set up the calculation table

Table

The table shows:

Raw data: $x$ and $y$ values
Deviations from the mean: $(x - \bar{x})$ and $(y - \bar{y})$
Squared deviations: $(x - \bar{x})^2$ and $(y - \bar{y})^2$
Products of deviations: $(x - \bar{x})(y - \bar{y})$

Step 2: Calculate the means

$\bar{x} = \frac{\sum x}{n} = \frac{192}{12} = 16$

$\bar{y} = \frac{\sum y}{n} = \frac{36}{12} = 3$

Step 3: Calculate Pearson's correlation coefficient

Using the formula:

$r = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sqrt{\sum (x - \bar{x})^2 \sum (y - \bar{y})^2}}$

Substituting the values from our table:

$r = \frac{-28}{\sqrt{594 \times 20}}$

$r \approx -0.25689$

$r \approx -0.26$

Step 4: Calculate the regression line

For the gradient:

$m = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sum (x - \bar{x})^2}$

$m = \frac{-28}{594}$

$m \approx -0.04714$

For the $y$ -intercept:

$b = \bar{y} - m\bar{x}$

$b = 3 + \frac{28 \times 16}{594}$

$b \approx 3.75421$

Final Results:

Correlation: $r = -0.26$ (weak negative correlation)
Line of best fit: $y = -0.047x + 3.754$

Classifying correlations

When describing correlation strength, we use verbal descriptions based on the value of $r$ . While there are no strict rules, the following classification provides helpful guidance.

For positive correlations (and similarly for negative correlations):

Correlation	Classification
$0.6$ to $1.0$	Strong
$0.4$ to $0.6$	Moderate
$0.1$ to $0.4$	Weak
$0.0$ to $0.1$	Virtually none

chatImportant

Important Points About Classification:

These categories are suggestions, not fixed rules
Different disciplines may use different criteria
Always have the scatterplot available when interpreting correlation
Be aware of any outliers that might affect the correlation
Think about the context of the data and what makes sense

Example: In our caller waiting times example, $r \approx -0.26$ indicates weak negative correlation. If an outlier were removed, the correlation becomes approximately $-0.57$ , which would be classified as moderate negative correlation.

bookmarkSummary

Key Formulae Summary

Pearson's correlation coefficient:

$r = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sqrt{\sum (x - \bar{x})^2 \sum (y - \bar{y})^2}}$

Least squares regression line: $y = mx + b$ , where

$m = \frac{\sum (x - \bar{x})(y - \bar{y})}{\sum (x - \bar{x})^2}$

and

$b = \bar{y} - m\bar{x}$

bookmarkSummary

Key Points to Remember:

Pearson's correlation coefficient $r$ measures linear correlation and is always a pure number between $-1$ and $1$
The numerator in the correlation formula involves products of deviations, showing whether variables work together or against each other
The least squares regression line minimises the sum of squared vertical distances from points to the line
The regression line always passes through the point $(\bar{x}, \bar{y})$
Once you've calculated $r$ , finding the regression line is quick because both formulae share the same numerator
Use the classification table as a guide, but always consider the scatterplot and context when interpreting correlation strength

Formulae for Correlation and Regression (HSC SSCE Mathematics Advanced): Revision Notes