Type I & Type II Errors Revision Notes for Edexcel A-Level Further Mathematics

21.3.1 Type I & Type II Errors

Introduction

In hypothesis testing, we make decisions about whether to reject or fail to reject the null hypothesis ( $H_0$ ) based on sample data. However, there is always a risk of making an error. These errors are classified as:

Type I Error: Rejecting $H_0$ when it is true.
Type II Error: Failing to reject $H_0$ when it is false. The probabilities of these errors help assess the effectiveness of a statistical test, often summarised using the power function.

Type I and Type II Errors

Type I Error:

Occurs when $H_0$ is true, but we reject it.
Probability of a Type I error is the significance level ( $\alpha$ ):

\alpha = P(\text{Type I Error}) = P(\text{Reject } H_0 | H_0 \text{ true})

Type II Error:

Occurs when $H_0$ is false, but we fail to reject it.
Probability of a Type II error is denoted by $\beta$ :

\beta = P(\text{Type II Error}) = P(\text{Fail to Reject } H_0 | H_1 \text{ true})

Power of a Test:

The power of a test is the probability of correctly rejecting $H_0$ when $H_1$ is true:

\text{Power} = 1 - \beta

Critical Region and Errors

The critical region determines when $H_0$ $H_{0}$ is rejected. Its size directly affects $\alpha$ $α$ and $\beta$ $β$ :
- A larger critical region increases $\alpha$ but decreases $\beta$
- A smaller critical region decreases $\alpha$ but increases $\beta$

Power Function

The power function of a test, $\pi(\theta)$ , gives the probability of rejecting $H_0$ for a specific value of the parameter $\theta$ under $H_1$ .

For a test statistic $X$ :

\pi(\theta) = P(\text{Reject } H_0 | \theta)

The power function helps visualise how effective the test is across different possible values of $\theta$ .

Worked Examples

Note

Example 1: Calculating Type I Error

Problem

A machine produces items with a mean weight $\mu$

We test $H_0: \mu = 50$ vs $H_1: \mu \neq 50$ at a significance level of 5%.

A sample mean of $\bar{x}$ is used, and the critical region is $|\bar{x} - 50| > 1.96$ (assuming standard normal distribution).

Calculate the probability of a Type $I$ error.

Step 1: Define Type I Error

A Type $I$ error occurs when $H_0$ is rejected even though $\mu = 50$ (i.e., $H_0$ is true)

Step 2: Use the Critical Region

Under $H_0: \mu = 50, \bar{x}$ follows a standard normal distribution:

Z = \frac{\bar{x} - 50}{\sigma / \sqrt{n}}

The critical region is:

|\bar{x} - 50| > 1.96 \quad \Rightarrow \quad |Z| > 1.96

Step 3: Find the Probability

For a standard normal distribution:

P(|Z| > 1.96) = P(Z > 1.96) + P(Z < -1.96)

Using symmetry:

P(Z > 1.96) = 0.025, \quad P(Z < -1.96) = 0.025

Thus:

P(|Z| > 1.96) = 0.025 + 0.025 = 0.05

Final Answer:

The probability of a Type $I$ error ( $\alpha$ ) is 0.05 or 5%

Note

Example 2: Calculating Type II Error and Power

Problem

Using the same test as in Example $1$ , calculate the probability of a Type $II$ error ( $\beta$ ) if $\mu = 49.5$ under $H_1: \mu \neq 50$ .

Assume $\sigma = 1$ and $n = 25$

Step 1: Define Type $II$ Error

A Type $II$ error occurs when $H_0$ is not rejected even though $H_1: \mu = 49.5$ is true.

This happens when $\bar{x}$ does not fall in the critical region:

-1.96 \leq Z \leq 1.96

Step 2: Find the Distribution Under $H_1$

If $\mu = 49.5$

Z = \frac{\bar{x} - 50}{\sigma / \sqrt{n}}

= \frac{\bar{x} - 50}{1 / \sqrt{25}} = 5(\bar{x} - 50)

For $\bar{x} \sim N(49.5, 0.2^2), Z \sim N\left(-2.5, 1\right)$

Step 3: Find $P(-1.96 \leq Z \leq 1.96)$

Standardise $Z$ under $H_1$ :

P(-1.96 \leq Z \leq 1.96) = P\left(\frac{-1.96 - (-2.5)}{1} \leq Z^* \leq \frac{1.96 - (-2.5)}{1}\right)

where $Z^* \sim N(0, 1)$

Simplify:

P(-1.96 \leq Z \leq 1.96) = P(0.54 \leq Z^* \leq 4.46)

From standard normal tables:

P(0.54 \leq Z^* \leq 4.46) = P(Z^* \leq 4.46) - P(Z^* \leq 0.54)

Using normal tables:

P(Z^* \leq 4.46) \approx 1, \quad P(Z^* \leq 0.54) \approx 0.7054

Thus:

P(-1.96 \leq Z \leq 1.96) \approx 1 - 0.7054 = 0.2946

Step 4: Calculate Power

The power is:

\text{Power} = 1 - \beta = 1 - 0.2946 = 0.7054

Final Answer:

Probability of Type $II$ error ( $\beta$ ): 0.2946
Power of the test: 0.7054

Note Summary

Note

Common Mistakes

Confusing Type I and Type II errors:

Type I: Reject $H_0$ when true.
Type II: Fail to reject $H_0$ when false.

Incorrect critical region: Ensure the critical region corresponds to the significance level.
Mixing distributions: Use the correct distribution under $H_0$ or $H_1$
Forgetting to calculate power: Power is the complement of $\beta$

Note

Key Formulas

Type $I$ Error:

\alpha = P(\text{Reject } H_0 | H_0 \text{ true})

Type $II$ Error:

\beta = P(\text{Fail to Reject } H_0 | H_1 \text{ true})

Power:

\text{Power} = 1 - \beta

Critical Region:

One-tailed: $P(X > c | H_0) \leq \alpha$
Two-tailed: Split $\alpha$ equally between tails.

Type I & Type II Errors (Edexcel A-Level Further Mathematics): Revision Notes

21.3.1 Type I & Type II Errors

Introduction

Type I and Type II Errors

Type I Error:

Type II Error:

Power of a Test:

Critical Region and Errors

Power Function

Worked Examples

Example 1: Calculating Type I Error

Example 2: Calculating Type II Error and Power

Note Summary

Common Mistakes

Key Formulas

Explore Edexcel A-Level Further Mathematics Model Answers by Topics

Poisson & Geometric Hypothesis Testing

Chi Squared Tests

Quality of Tests

Explore Edexcel A-Level Further Mathematics Quizzes by Topics

Poisson & Geometric Hypothesis Testing

Chi Squared Tests

Quality of Tests

Explore Edexcel A-Level Further Mathematics Flashcards by Topics

Poisson & Geometric Hypothesis Testing

Chi Squared Tests

Quality of Tests

Explore Edexcel A-Level Further Mathematics Exam Questions by Topics

Poisson & Geometric Hypothesis Testing

Chi Squared Tests

Quality of Tests

Join 100,000+ A-Level students studying Revision Notes with us.