Theorem Weak Law of Large Numbers

Weak Law of Large Numbers Let \(X_1, X_2, \ldots, X_n\) be a random sample from a population that has finite mean \(\mu\) and finite variance \(\sigma^2\). Then, the sequence of sample means \(\bar{X}_n=\sum_{i=1}^n \frac{X_i}{n}\) converges in probability to the population mean \(\mu\).

Proof:

The idea is to apply Chebyshev’s Inequality to \(\bar{X}_n\). In order to do this, we need to determine \(E\left(\bar{X}_n\right)\) and \(V\left(\bar{X}_n\right)\). Note that since \(X_1, \ldots, X_n\) is a random sample from a population with mean \(\mu\) and variance \(\sigma^2\), it follows that, \[ E\left(X_i\right)=\mu, \quad V\left(X_i\right)=\sigma^2 \quad \text { for all } i=1,2, \ldots, n \]

We can now calculate the mean and variance if \(\bar{X}_n\). First, we have that \[ E\left[\bar{X}_n\right]=E\left(\sum_{i=1}^n \frac{X_i}{n}\right)=\frac{1}{n} \sum_{i=1}^n E\left(X_i\right)=\frac{n \mu}{n}=\mu \]

and, since the \(X_i\) ’s are independent, we have \[ V\left(\bar{X}_n\right)=V\left(\sum_{i=1}^n \frac{X_i}{n}\right)=\frac{1}{n^2} \sum_{i=1}^n V\left(X_i\right)=\frac{n \sigma^2}{n^2}=\frac{\sigma^2}{n} . \]

We can now apply Chebyshev’s Inequality. Fix \(\varepsilon>0\) and set \(k=\frac{\sqrt{n} \varepsilon}{\sigma}\). Then, Chebyshev’s Inequality applied to \(\bar{X}_n\) implies that, \[ P\left(\left|\bar{X}_n-\mu\right|<\frac{\sqrt{n} \varepsilon}{\sigma} \cdot \frac{\sigma}{\sqrt{n}}\right) \geq 1-\frac{1}{(\sqrt{n} \varepsilon / \sigma)^2} \Longrightarrow P\left(\left|\bar{X}_n-\mu\right|<\varepsilon\right) \geq 1-\frac{\sigma}{n \varepsilon^2} \text { for all } n \geq 1 \]

Since this holds for all \(n \geq 1\), we are permitted to let \(n \rightarrow \infty\). This yields, \[ \lim _{n \rightarrow \infty} P\left(\left|\bar{X}_n-\mu\right|<\varepsilon\right) \geq 1 \Longrightarrow \lim _{n \rightarrow \infty} P\left(\left|\bar{X}_n-\mu\right|<\varepsilon\right)=1 \]

Since \(\varepsilon>0\) was fixed arbitrarily, this shows \(\bar{X}_n \xrightarrow{p} \mu\).