Probability

Law of Large Numbers

Sample mean convergence and concentration inequality comparison

Convergence with concentration bounds

Tail probability decay

Comparing the bounds

  • Markov only uses $\mathbb{E}[X]$. Loosest bound, often doesn't shrink with n for the sample mean.
  • Chebyshev uses variance. Decays as $\frac{\sigma^2}{n\varepsilon^2}$, tighter but still polynomial.
  • Chernoff uses bounded range. Decays as $e^{-2n\varepsilon^2 / R^2}$, exponentially fast. Only for bounded distributions.
  • $M_n \to \mu$ with probability 1 as $n \to \infty$. All bounds confirm this, at different speeds.