Outliers & Cleaning Data (AQA A-Level Mathematics): Revision Notes

Example: Consider the data set:

$2, 5, 7, 8, 10, 12, 15, 18, 20, 100$

• Q1 (Lower Quartile) = $6.5$
• Q3 (Upper Quartile) = $17$
• IQR = $17 - 6.5 = 10.5$
• Lower Bound = $6.5 - (1.5 × 10.5) = -9.25$ (No outliers on the lower side)
• Upper Bound = $17 + (1.5 × 10.5) = 32.75$ Here, $100$ is an outlier because it is greater than $32.75$.

Example: Cleaning Data Suppose you have the following data set of test scores:

$85, 90, 95, 100, 110, 700$

Step 1: Identify Outliers:

The score $700$ is an outlier.

Step 2: Handle the Outlier:

Investigate the cause. If it's a data entry error, correct or remove it.

Step 3: Handle Missing Data:

If you had a missing value in the test scores, you might replace it with the mean score or use another method.

Step 4: Check for Consistency:

Ensure all scores are within the expected range ($0$ to $100$).

Cleaning your data ensures that your analysis is based on accurate and reliable data, leading to more trustworthy results.

An outlier is an item of data that lies:

• $2$ standard deviations from the mean.
• $1.5$ interquartile ranges from the median.

Example: Cleaning Data with Standard Deivation Let's go through a detailed example to understand how to formally identify outliers using standard deviation.

Question: Using standard deviation, formally identify any outliers in the following set: $1.3, 2.4, 6.7, 2.8, 3.9, 0.1$

Step 1: Calculate the Mean and Standard Deviation

Using a calculator (shown below), we can find the mean and standard deviation of the data set.

From the calculator screen, we have:

• Mean ($\bar{x}$) = $2.867$
• Standard deviation (σ) = $2.085$

Step 2: Determine the Outlier Boundaries

Outliers are defined as data points that lie outside two standard deviations from the mean.

We calculate the boundaries for the outliers using the formula: $ˉ ±2σ$

Substitute the values of the mean ($\bar{x}$) and standard deviation (σ)

Thus, the boundaries for outliers are: $−1.303$ and $7.037$

Step 3: Identify the Outliers

Any data points that fall outside the range $[−1.303,7.037]$ are considered outliers.

Checking the data set:

$1.3, 2.4, 6.7, 2.8, 3.9, 0.1$

All of these values lie within the range $[−1.303,7.037]$, so there are no outliers in this data set.

Explanation:

Since no data points lie outside the boundaries of$[−1.303,7.037]$, we conclude that this data set has no outliers.

Example: Identifying Outliers using the IQR Let's go through a detailed example to understand how to identify outliers using the Interquartile Range (IQR).

Question: Using the same data set as before, identify any outliers using the IQR method: $1.3, 2.4, 6.7, 2.8, 3.9, 0.1$

Step 1: Calculate the Median and IQR

The median and quartiles can be found using a calculator. Here's the result:

From the calculator screen, we have:

• Median ($Med$) = $2.6$
• Lower quartile ($Q_1$) = $1.3$
• Upper quartile ($Q_3$) = $3.9$

Thus, the IQR (Interquartile Range) is:

Step 2: Determine the Outlier Boundaries

Outliers are defined as any data points that lie 1.5 times the IQR above $Q_3$ or below $Q_1$.

We calculate the boundaries for outliers using the formula:

$Med±1.5×IQR$

Substitute the values:

Thus, the outlier boundaries are:

$[-1.3, 6.5]$

Step 3: Identify the Outliers

Any data points that fall outside the range $[-1.3, 6.5]$ are considered outliers.

Checking the data set:

$1.3, 2.4, 6.7, 2.8, 3.9, 0.1$

The value $6.7$ lies outside this range (greater than $6.5$), so $6.7$ is an outlier.

Explanation:

Since $6.7$ lies outside the interval $[-1.3, 6.5]$, we can conclude that $6.7$ is an outlier in this data set.