From binomial to Poisson: the law of rare events
Many small loans that each might default, thousands of characters on a page that each might be a typo, many requests to a server that each might fail: when there are many independent chances and each one is rare, how is the number of events distributed?
To a close approximation, the distribution depends only on the expected number of events, not on how many chances there are or how rare each one is. Stage 1 asks you to estimate one such probability before the page computes it.
Before you start · 3 quick checks on the binomial setup, binomial coefficients and the expected count, about 4 min
The binomial setup. Run \(n\) trials. Assume that they are independent and that each one is a success with the same probability \(p\), where \(0 \le p \le 1\). The number \(X\) of successes is a binomial random variable with parameters \(n\) and \(p\). It takes the values \(0, 1, \dots, n\), with
\[P(X = k) = \binom{n}{k} p^{k} (1 - p)^{n-k},\]and its expectation is \(E[X] = np\). The lesson on PMF, PDF and CDF drew probability mass functions like this one as bars, and the lesson on expectation and variance defined \(E[X]\).
Rare events. This page is about binomial counts with many trials and a small \(p\), while the expected count \(np\) is moderate. A success is then whatever is being counted, often something unwanted such as a default, a typo or a failed request, and it is called an event.
Fine print: what independence means here
Independence means that learning the outcomes of some trials does not change the probabilities for the others. Real loans, typos and requests satisfy this only approximately: a recession raises many default probabilities at once, and a tired typist makes typos in clusters. Each example on this page states independence as part of a model, and its numbers describe that model.
Guess the chance of no event
How the guess works
Type a decimal, a fraction or a percentage, then press Commit my guess (or Enter). The page then shows the exact value and a close approximation, says what your number corresponds to, and opens a chart in which you can change the number of trials while the expected number of events stays fixed. Guess again clears all of it.
Grow the number of trials
How close, and when?
Two predictions first. The explorer below them opens once both are answered.
Explorer: three values of the mean
Here \(X\) is binomial with \(n\) trials and \(p = \lambda/n\), and \(Y\) is Poisson with the same \(\lambda\). Choose \(\lambda\), then grow \(n\). The tiles add the binomial variance, and the readout adds Le Cam's bound from the box below.
Goals
Mean and variance of a Poisson count. Let \(Y\) be Poisson with parameter \(\lambda\). The term with \(k\) equal to 0 adds nothing to the mean, and \(\frac{k}{k!} = \frac{1}{(k-1)!}\) for \(k\) of 1 or more, so
\[\begin{aligned} E[Y] &= \sum_{k=1}^{\infty} k\,\frac{\lambda^{k} e^{-\lambda}}{k!} \\ &= \lambda \sum_{k=1}^{\infty} \frac{\lambda^{k-1} e^{-\lambda}}{(k-1)!} = \lambda, \end{aligned}\]because the last sum adds all the Poisson probabilities. The same step, done twice, gives \(E[Y(Y - 1)] = \lambda^{2}\), and then
\[\begin{aligned} \operatorname{Var}(Y) &= E[Y(Y - 1)] + E[Y] \\ &\quad - (E[Y])^{2} \\ &= \lambda^{2} + \lambda - \lambda^{2} = \lambda. \end{aligned}\]A binomial count with \(p = \lambda/n\) has mean \(\lambda\) and variance \(\lambda(1 - \lambda/n)\), which tends to \(\lambda\).
The Poisson limit theorem
Assumptions. Fix a number \(\lambda \gt 0\) and an integer \(k \ge 0\). For each integer \(n \gt \lambda\), let \(X_n\) count the successes in \(n\) independent trials, each a success with probability \(p_n = \lambda/n\), so that \(E[X_n] = np_n = \lambda\) for every \(n\).
Given. The binomial PMF from the setup, with \(p_n = \lambda/n\), is the starting formula: for every integer \(n \ge k\) with \(n \gt \lambda\),
\[\begin{aligned} &P(X_n = k) \\ &= \binom{n}{k}\left(\frac{\lambda}{n}\right)^{k}\left(1 - \frac{\lambda}{n}\right)^{n-k}. \end{aligned}\]Conclusion. As \(n \to \infty\) with \(np_n = \lambda\) held fixed,
\[\lim_{n \to \infty} P(X_n = k) = \frac{\lambda^{k} e^{-\lambda}}{k!}.\]Facts the proof uses.
- the starting formula above, which is the binomial PMF;
- from calculus, \(\left(1 + \frac{x}{n}\right)^{n} \to e^{x}\) as \(n \to \infty\), for every number \(x\);
- the product rule for limits: if the number of sequences does not change with \(n\) and each has a finite limit, their product tends to the product of the limits; and if \(a_n \to a\) with \(a \gt 0\), then \(a_n^{m} \to a^{m}\) for every fixed integer \(m\).
Some options state a law for every \(k \ge 1\) or for every \(\lambda \gt 0\); such a line must hold over that whole range. Pick the line that holds in every case the theorem covers, not one that is right only for some values of \(\lambda\), \(k\) or \(n\).
- The case \(k\) equal to 0 is stage 1: the probability of no event, \(\left(1 - \frac{\lambda}{n}\right)^{n}\), tends to \(e^{-\lambda}\).
- The limits form a whole distribution: the Poisson probabilities add to 1, and the Poisson mean and variance, both \(\lambda\), are the limits of the binomial mean \(\lambda\) and variance \(\lambda(1 - \lambda/n)\).
- The same limit holds when \(np_n\) only tends to \(\lambda\) instead of equaling it, by a similar argument.
- The theorem says where the binomial probabilities go, not how fast. Le Cam's inequality from stage 2 adds a rate: every gap is at most \(np_n^{2} = \lambda^{2}/n\).
Problems
Each problem states its model, and has hints and a full solution.
More problems
Further reading
- J. K. Blitzstein and J. Hwang, Introduction to Probability, 2nd ed. (CRC Press, 2019), ch. 4, "Expectation," sections 4.7 and 4.8 on the Poisson distribution and its connection with the binomial.
- S. Ross, A First Course in Probability, 10th ed. (Pearson, 2019), ch. 4, "Random Variables," section 4.7 on the Poisson random variable.
- W. Feller, An Introduction to Probability Theory and Its Applications, vol. 1, 3rd ed. (Wiley, 1968), ch. 6, "The Binomial and the Poisson Distributions."
- L. Le Cam, "An approximation theorem for the Poisson binomial distribution," Pacific Journal of Mathematics 10(4) (1960), 1181–1197, on the bound used in stage 2.