Probability
Law of Large Numbers
Sample mean convergence and concentration inequality comparison
Convergence with concentration bounds
Tail probability decay
Comparing the bounds
- Markov only uses $\mathbb{E}[X]$. Loosest bound, often doesn't shrink with n for the sample mean.
- Chebyshev uses variance. Decays as $\frac{\sigma^2}{n\varepsilon^2}$, tighter but still polynomial.
- Chernoff uses bounded range. Decays as $e^{-2n\varepsilon^2 / R^2}$, exponentially fast. Only for bounded distributions.
- $M_n \to \mu$ with probability 1 as $n \to \infty$. All bounds confirm this, at different speeds.