Proofs for the Expectation and Variance (VCE SSCE Mathematical Methods): Revision Notes

Proofs for the Expectation and Variance

This note presents rigorous mathematical proofs of three fundamental results about the binomial distribution. Understanding these proofs will deepen your knowledge of why the expected value and variance formulas work.

Introduction

The binomial distribution has several important properties that can be proven mathematically. In this note, we establish three key results:

1. All probabilities in a binomial distribution sum to 1
2. The expected value is E(X) = np
3. The variance is Var(X) = np(1-p)

Each proof uses algebraic manipulation and the binomial theorem as a foundation.

Proof 1: The probabilities sum to 1

Result: The probabilities of a binomial distribution sum to 1.

The binomial theorem

The binomial theorem states that:

$$(a + b)^n = \sum_{k=0}^{n} \binom{n}{k} a^{n-k} b^k$$

This theorem is the key tool we use to prove that probabilities sum to 1.

Proof:

For a binomial random variable $X$ with parameters $n$ and $p$, we need to show:

$$\sum_{x=0}^{n} \Pr(X = x) = 1$$

Starting with the left-hand side:

$$\sum_{x=0}^{n} \Pr(X = x) = \sum_{x=0}^{n} \binom{n}{x} p^x (1-p)^{n-x}$$

Notice that this sum has the same form as the binomial theorem with $a = (1-p)$ and $b = p$:

$$= [(1-p) + p]^n$$ $$= (1)^n$$ $$= 1$$

This confirms that all the probabilities in a binomial distribution sum to 1, as required.

Proof 2: Expected value

Result: If $X$ is a binomial random variable with parameters $n$ and $p$, then $E(X) = np$.

Proof:

By the definition of expected value:

$$E(X) = \sum_{x=0}^{n} x \cdot \binom{n}{x} p^x (1-p)^{n-x}$$

Expand the binomial coefficient:

$$= \sum_{x=0}^{n} x \cdot \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x}$$

Notice that when $x = 0$, the term equals zero, so we can start the sum from $x = 1$:

$$= \sum_{x=1}^{n} x \cdot \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x}$$

Since $x! = x(x-1)!$, we can write:

$$= \sum_{x=1}^{n} x \cdot \frac{n!}{x(x-1)!(n-x)!} p^x (1-p)^{n-x}$$

Cancel the $x$ terms:

$$= \sum_{x=1}^{n} \frac{n!}{(x-1)!(n-x)!} p^x (1-p)^{n-x}$$

Key strategy: Factor out n and p from the expression.

$$= np \sum_{x=1}^{n} \binom{n-1}{x-1} p^{x-1} (1-p)^{n-x}$$

Let $z = x - 1$. When $x = 1$, $z = 0$; when $x = n$, $z = n-1$:

$$= np \sum_{z=0}^{n-1} \binom{n-1}{z} p^z (1-p)^{n-1-z}$$ $$E(X) = np \times 1 = np$$

This completes the proof that the expected value of a binomial distribution is $np$.

Proof 3: Variance

Result: If $X$ is a binomial random variable with parameters $n$ and $p$, then $\text{Var}(X) = np(1-p)$.

Proof:

We use the formula: $\text{Var}(X) = E(X^2) - \mu^2$, where $\mu = np$

To find the variance, we first need to determine $E(X^2)$:

$$E(X^2) = \sum_{x=0}^{n} x^2 \binom{n}{x} p^x (1-p)^{n-x}$$ $$= \sum_{x=0}^{n} x^2 \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x}$$

Problem: The term $x^2$ is not a factor of $x!$, so we cannot proceed as in the previous proof.

Strategy: Instead, we determine E[X(X-1)] first:

$$E[X(X-1)] = \sum_{x=0}^{n} x(x-1) \binom{n}{x} p^x (1-p)^{n-x}$$ $$= \sum_{x=0}^{n} x(x-1) \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x}$$

When $x = 0$ or $x = 1$, the term $x(x-1) = 0$, so we start from $x = 2$:

$$= \sum_{x=2}^{n} x(x-1) \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x}$$

Factor out $n(n-1)p^2$ and let $z = x - 2$:

$$= n(n-1)p^2 \sum_{x=2}^{n} \frac{(n-2)!}{(x-2)!(n-x)!} p^{x-2} (1-p)^{n-x}$$ $$= n(n-1)p^2 \sum_{z=0}^{n-2} \binom{n-2}{z} p^z (1-p)^{n-2-z}$$

This sum represents all probabilities for a binomial random variable $Z$ with $n-2$ trials and probability $p$, which equals 1.

Therefore:

$$E[X(X-1)] = n(n-1)p^2$$

Now we can find $E(X^2)$. Since $E[X(X-1)] = E(X^2) - E(X)$:

$$E(X^2) - E(X) = n(n-1)p^2$$ $$E(X^2) = n(n-1)p^2 + E(X)$$ $$E(X^2) = n(n-1)p^2 + np$$

This gives us $E(X^2)$ in terms of $n$ and $p$. Now we can calculate the variance:

$$\text{Var}(X) = E(X^2) - \mu^2$$ $$= n(n-1)p^2 + np - (np)^2$$ $$= n^2p^2 - np^2 + np - n^2p^2$$ $$= np - np^2$$ $$= np(1-p)$$

This completes the proof that $\text{Var}(X) = np(1-p)$.