Mathematics
The Law of Large Numbers and Its Intuitive Meaning
Quick fact
In a large-scale simulation of coin flips, the proportion of heads converges to 0.5, but the absolute difference from 0.5 can actually grow as the number of flips increases—yet the proportion gets closer to 0.5.
Why this is interesting
Have you ever wondered why flipping a coin 100 times almost always gives you close to 50 heads and 50 tails, even though a short streak of heads might occur? Why does the average of random events become so predictable over time?
Read the full explanation
Understanding The Law of Large Numbers and Its Intuitive Meaning
Imagine you are rolling a fair six-sided die. The expected average is 3.5. If you roll it only a few times, your average might be far off, like 2 or 5. But as you roll it thousands of times, the average will get closer and closer to 3.5. This is the law of large numbers: the more trials you do, the more the observed average converges to the true expected value. Think of it as a balancing act: individual outcomes are unpredictable, but the combined effect of many outcomes tends to cancel out the extremes and produce a stable average. For example, if you toss a coin, the proportion of heads will approach 0.5 as you toss more and more, even though the total number of heads might not be exactly half. The key is that the fluctuation around the expected value shrinks relative to the number of trials.
A deeper explanation
The law of large numbers works because the variance of the sample mean decreases with the sample size. If you have independent and identically distributed trials with a finite expected value and variance, the sample mean (the average of all outcomes) has a mean equal to the expected value and a variance equal to the population variance divided by the sample size. As the sample size grows, the variance of the sample mean shrinks to zero, meaning the probability that the sample mean is far from the expected value becomes extremely small. Formally, the weak law of large numbers states that the sample mean converges in probability to the expected value, while the strong law states it converges almost surely. This principle is why insurance companies can predict total claims, why pollsters can estimate public opinion with a margin of error, and why casinos always win in the long run. It also explains why the gambler's fallacy is a misconception: past outcomes do not affect future ones, but the law of large numbers ensures that long-run frequencies stabilize. The concept is foundational for statistical inference, as it justifies using sample averages to estimate population parameters.