The exponential distribution and memorylessness
You have already waited 10 minutes for a bus. Is the chance that it comes in the next 5 minutes higher, the same, or lower than the chance that it would have come in the first 5 minutes?
The answer is a conditional probability, so it depends on the model for the waiting time. Stage 1 asks you to commit to an answer for three models before the page computes anything.
Before you start · 3 quick checks on a density and its CDF, conditional probability and the function e^−x, about 4 min
Waiting times. A waiting time is a random variable \(T\) with \(T \ge 0\). When \(T\) has a density \(f\), its CDF is \(F(t) = P(T \le t)\), the area under \(f\) from 0 to \(t\), and the probability that the wait lasts longer than \(t\) is
\[P(T \gt t) = 1 - F(t).\]The lesson on PMF, PDF and CDF introduced densities and CDFs, and the lesson on expectation and variance defined \(E[T]\) and \(\operatorname{Var}(T)\).
The exponential distribution. Fix a number \(\lambda \gt 0\), the rate. \(T\) is exponential with rate \(\lambda\) when its density is \(f(t) = \lambda e^{-\lambda t}\) for \(t \ge 0\) and \(f(t) = 0\) for \(t \lt 0\). Then for every \(t \ge 0\)
\[\begin{aligned} F(t) &= \int_{0}^{t} \lambda e^{-\lambda x}\,dx = 1 - e^{-\lambda t}, \\ P(T \gt t) &= e^{-\lambda t}. \end{aligned}\]Its mean is \(E[T] = 1/\lambda\) (worked out in stage 2), so a mean wait of 10 minutes means \(\lambda = 1/10\) per minute.
Fine print: units, and why the total area is 1
The rate \(\lambda\) is measured per unit of time, and \(\lambda t\) has no unit. A rate of 0.5 per minute is a rate of 30 per hour; using a rate per minute with a time in hours gives a wrong answer.
The density has total area 1: the area from 0 to \(t\) is \(1 - e^{-\lambda t}\), which tends to 1 as \(t\) grows, because \(e^{-\lambda t}\) tends to 0.
Does waiting change the odds?
How the answers work
Click one answer for each model. An answer is locked once clicked, and nothing about any answer is shown until all three are in. Then the page marks each answer, explains it, and opens figures with a slider for the time already waited. Answer again clears all of it.
Slide the time already waited
The figures and the table follow the slider. The first figure shows, for each model, the chance of a bus in the next 5 minutes after a wait of \(s\) minutes; the three panels below it show where that chance comes from.
| Exponential | Uniform | Two-stage |
|---|
The formulas behind the table
How long is a typical wait?
Two predictions first. The explorer below them opens once both are answered.
Explorer: the survival function
Choose the mean \(1/\lambda\) and move \(t\). The curve is \(P(T \gt t) = e^{-\lambda t}\), the probability that the wait lasts longer than \(t\).
Goals
Mean. Let \(T\) be exponential with rate \(\lambda\). Integrating by parts,
\[\begin{aligned} E[T] &= \int_{0}^{\infty} t\,\lambda e^{-\lambda t}\,dt \\ &= \Big[-t e^{-\lambda t}\Big]_{0}^{\infty} + \int_{0}^{\infty} e^{-\lambda t}\,dt \\ &= 0 + \frac{1}{\lambda}. \end{aligned}\]Spread. The same step done twice gives \(E[T^{2}] = 2/\lambda^{2}\), so
\[\begin{aligned} \operatorname{Var}(T) &= E[T^{2}] - (E[T])^{2} \\ &= \frac{2}{\lambda^{2}} - \frac{1}{\lambda^{2}} = \frac{1}{\lambda^{2}}, \end{aligned}\]and \(\operatorname{SD}(T) = 1/\lambda\), equal to the mean.
Median. \(1 - e^{-\lambda m} = \tfrac{1}{2}\) gives \(m = (\ln 2)/\lambda\). Since \(\ln 2 \approx 0.6931\) is less than 1, the median is below the mean.
Hazard rate. For a waiting time with density \(f\), the hazard rate at a time \(t\) with \(P(T \gt t) \gt 0\) is \(h(t) = f(t)/P(T \gt t)\): roughly, the rate of arrival among the waits still going on at \(t\). For the exponential, \(h(t) = \lambda e^{-\lambda t}/e^{-\lambda t} = \lambda\) at every \(t \ge 0\), a constant. For every \(\Delta \gt 0\),
\[\begin{aligned} &P(t \lt T \le t + \Delta \mid T \gt t) \\ &= 1 - e^{-\lambda \Delta}, \end{aligned}\]which is at most \(\lambda \Delta\) and close to \(\lambda \Delta\) when \(\Delta\) is small (the lesson on Taylor polynomials makes "close" precise). The other two models of stage 1 have hazard rates that rise with \(t\): \(1/(20 - t)\) for the uniform model when \(0 \le t \lt 20\), and \(t/(5(t + 5))\) for the two-stage model.
The Poisson process. Events such as calls or arrivals form a Poisson process with rate \(\lambda\) when (i) the numbers of events in time intervals that do not overlap are independent, and (ii) the number of events in any interval of length \(t\) has the Poisson distribution with mean \(\lambda t\). The lesson on the law of rare events shows where (ii) comes from: many short time slots, each with a small chance of an event. Let \(N(t)\) be the number of events in \([0, t]\) and \(T_1\) the time of the first event. The first event comes after \(t\) exactly when \([0, t]\) holds no event, so
\[P(T_1 \gt t) = P(N(t) = 0) = e^{-\lambda t},\]and \(T_1\) is exponential with rate \(\lambda\). The time \(T_2\) of the second event comes after \(t\) exactly when \([0, t]\) holds at most one event:
\[\begin{aligned} P(T_2 \gt t) &= P(N(t) \le 1) \\ &= e^{-\lambda t}(1 + \lambda t). \end{aligned}\]With \(\lambda = 1/5\) per minute this is the two-stage model of stage 1.
The exponential distribution is memoryless
Assumptions. Fix \(\lambda \gt 0\), and let \(T\) be exponential with rate \(\lambda\), so that \(F(u) = P(T \le u) = 1 - e^{-\lambda u}\) for every \(u \ge 0\). Let \(s \ge 0\) and \(t \ge 0\) be any two numbers; either may be 0.
Conclusion.
\[P(T \gt s + t \mid T \gt s) = P(T \gt t).\]Given that the wait has already lasted longer than \(s\), the chance that it lasts more than \(t\) longer is the chance that a fresh wait lasts more than \(t\).
Facts the proof uses.
- the complement rule: \(P(T \gt u) = 1 - P(T \le u)\);
- the definition of conditional probability: \(P(A \mid B) = P(A \cap B)/P(B)\) for events with \(P(B) \gt 0\);
- \(e^{a + b} = e^{a}e^{b}\) for all numbers \(a\) and \(b\), and \(e^{x} \gt 0\) for every number \(x\).
Each option is one sentence about the \(\lambda\), \(s\) and \(t\) of the theorem. Pick the one that is true in every case the theorem covers, \(s = 0\) and \(t = 0\) included.
- Constant hazard. For every \(s \ge 0\) and \(\Delta \gt 0\), the chance of an arrival in the next \(\Delta\) does not depend on \(s\): \[\begin{aligned} &P(s \lt T \le s + \Delta \mid T \gt s) \\ &= 1 - e^{-\lambda \Delta}. \end{aligned}\]
- Only the exponential. Let \(T \ge 0\) be any random variable with \(P(T \gt s) \gt 0\) for every \(s \ge 0\). If \(P(T \gt s + t \mid T \gt s) = P(T \gt t)\) for all \(s, t \ge 0\), then \(T\) is exponential, with rate \(\lambda = -\ln P(T \gt 1)\). The geometric distribution (the number of independent trials up to and including the first success, with values 1, 2, …) has the same property for whole numbers \(s\) and \(t\), but not for all real \(s, t \ge 0\).
- Poisson processes. For a Poisson process, the wait from any fixed time \(s\) to the next event is again exponential with rate \(\lambda\), whatever happened before \(s\): by assumption (i), the events in \([s, s + t]\) do not depend on those before \(s\), and by (ii) there are none with probability \(e^{-\lambda t}\).
Optional: why only the exponential is memoryless
Write \(G(u) = P(T \gt u)\), with \(G(u) \gt 0\) for every \(u \ge 0\). Since \(\{T \gt s + t\}\) lies inside \(\{T \gt s\}\), the property says \(G(s + t)/G(s) = G(t)\), that is, \(G(s + t) = G(s)G(t)\) for all \(s, t \ge 0\). With \(s = t = 0\) this gives \(G(0) = G(0)^{2}\), and \(G(0) \gt 0\), so \(G(0) = 1\).
Let \(a = G(1)\). For whole numbers \(m \ge 0\) and \(n \ge 1\), using the product rule again and again, \(G(m/n) = G(1/n)^{m}\) and \(a = G(1/n)^{n}\), so \(G(q) = a^{q}\) for every rational \(q \ge 0\). Here \(0 \lt a \le 1\). If \(a\) were 1, then \(P(T \gt q) = 1\) for every rational \(q\), which is impossible: \(P(T \gt q)\) tends to 0 as \(q\) grows, because \(T\) takes finite values. So \(0 \lt a \lt 1\), and with \(\lambda = -\ln a \gt 0\), \(G(q) = e^{-\lambda q}\) for every rational \(q \ge 0\).
The one regularity property used next holds for every random variable: \(G\) never increases, because a larger \(u\) gives a smaller event \(\{T \gt u\}\). For a real \(u \ge 0\) and rationals \(q_1 \le u \le q_2\), \(e^{-\lambda q_2} \le G(u) \le e^{-\lambda q_1}\), and letting \(q_1\) and \(q_2\) approach \(u\) gives \(G(u) = e^{-\lambda u}\). So \(F(u) = 1 - e^{-\lambda u}\) for every \(u \ge 0\): \(T\) is exponential with rate \(\lambda\).
Problems
Each problem states its model, and has hints and a full solution.
Further reading
- J. K. Blitzstein and J. Hwang, Introduction to Probability, 2nd ed. (CRC Press, 2019), ch. 5, "Continuous Random Variables," sections 5.5 and 5.6 on the exponential distribution and Poisson processes.
- S. Ross, A First Course in Probability, 10th ed. (Pearson, 2019), ch. 5, "Continuous Random Variables," section 5.5 on exponential random variables and hazard rate functions.
- J. C. Hull, Options, Futures, and Other Derivatives, 10th ed. (Pearson, 2018), ch. 24, "Credit Risk," on default probabilities from hazard rates.