Rare Events in Continuous Time: From Poisson Arrivals to Exponential Waiting Times

Rare events happen at particular moments: a customer reaches a desk, a detector records a click, or a component fails. We are going to build a continuous-time model from ordinary probability, beginning with nothing more mysterious than many small independent trials. Here is the question that will organize the first half. During a stretch of time of length T, how many events arrive? We want the complete distribution of that count, not merely its average. Picture time as a line. The red marks are arrivals from one possible run of the process. Another run would put them elsewhere and might contain a different number, but every arrival is attached to some moment. Now chop the observation time into n equal intervals. One interval has length delta t, equal to T divided by n. We will eventually make these intervals so short that an event inside one of them is genuinely rare. The model makes two substantive assumptions. First, separate intervals contribute independently. Second, the event rate is constant, so an interval of length delta t has event probability approximately lambda times delta t. Lambda is a rate, measured in events per unit time. Multiplying it by the interval length gives a dimensionless probability. As delta t shrinks, this probability shrinks too, but the total expected count across all n intervals remains lambda T. Each tiny interval is therefore a Bernoulli trial: event or no event. Adding n independent trials gives a binomial count X sub n, with n trials and success probability lambda T over n. Continuous time has not appeared by magic. We have built an approximation that can be pushed to a limit.

Before taking a limit, hold the central idea still. Suppose the expected count is two. Ten intervals can each have probability zero point two. One hundred intervals use zero point zero two, and one thousand use zero point zero zero two. The individual trials become rarer while their number grows. In every row, n times p sub n remains two. More generally, call the fixed expected count mu, equal to lambda T. Now fix a possible count k. The binomial probability of exactly k events is the number of ways to choose their intervals, times the probability of k occupied intervals, times the probability that all the others are empty. Substitute p sub n equal to mu over n and rearrange. The probability is mu to the k over k factorial, multiplied by three factors whose limits we can read separately. The first factor contains k terms: n over n, then n minus one over n, and so on. For a fixed k, every one of those terms approaches one. Therefore A sub n approaches one. The second factor is the classical exponential limit. One minus mu over n, raised to n, approaches e to the minus mu. This is the surviving probability of seeing no event across the complete interval. The last factor raises something approaching one to the fixed power minus k, so it also approaches one. Multiplying the three limits leaves e to the minus mu times mu to the k over k factorial. Finally replace mu by lambda T. This is the Poisson distribution, obtained rather than announced. Its probabilities really do sum to one, because the remaining series is the exponential series for e to the mu.

Here is a Poisson distribution with mu equal to three. The bars give the probabilities of zero events, one event, two events, and so on. The distribution is not symmetric, but it is concentrated around counts near three. Its average can be calculated exactly. Start with the definition of expectation: sum k times the probability of k. The factor k cancels one factor from k factorial. Pull one factor of mu outside, then re-index with j equal to k minus one. What remains is the sum of every Poisson probability, so that sum is one. The expected count is therefore mu. For variance, the useful quantity is the second factorial moment, N times N minus one. The same cancellation, now performed twice, gives mu squared. Ordinary variance can be written as the second factorial moment plus the mean, minus the square of the mean. Substitute mu squared, mu, and mu squared. The two squares cancel, leaving mu. So a Poisson count has mean mu and variance mu. The standard deviation is therefore the square root of mu. Relative fluctuations become smaller when the expected count becomes large, even though the absolute fluctuations grow. This equality has an intuitive origin in the small-interval model. Independent intervals each contribute a tiny yes-or-no uncertainty. Adding them gives variance n p times one minus p, which approaches n p as p becomes tiny. More importantly, equal mean and variance is a prediction that data can challenge. If repeated equal windows have variance far above their mean, arrivals may be clustered or the rate may be changing. If variance is far below the mean, some regular spacing or dead time may be present.

The count question has a natural companion. Instead of fixing a time interval and asking how many events occur, start the clock now and ask how long it runs before the next event. Call that random waiting time W. On one realized timeline, now is the green mark and the next arrival is the first red mark. The yellow span between them is the observed value of W. Another run would give a different span. To find the distribution, ask when W exceeds a chosen time t. That happens exactly when the interval from now to t contains zero events. Waiting longer than t and counting zero by t are the same event. But the count by time t is Poisson with mean lambda t. Put k equal to zero in the Poisson formula. The factorial and power disappear, leaving e to the minus lambda t. That gives the survival probability, the chance the wait is still not over. Its complement gives the cumulative distribution. Differentiating gives the exponential density, lambda e to the minus lambda t. For a unit rate, the survival curve begins at one. A very short threshold is likely to be crossed without an event, while a long event-free stretch becomes progressively less likely. Move the threshold to the right. The point rides down the curve because the probability of surviving without an event decays exponentially. The mean waiting time is the area under the survival curve. Integrating e to the minus lambda t from zero to infinity gives one over lambda. Double the event rate and the average wait is cut in half.

This is the claim people rightly distrust. Suppose we have already waited s units of time with no arrival. Surely, it feels, we must now be closer to the next event. The exponential model says no. Let us prove exactly what that sentence means. Begin at zero and suppose no event occurs before s. The yellow span is the waiting time we have already endured. Reaching s tells us that W is greater than s. Now ask for no event during an additional duration t. That means the total wait exceeds s plus t. The green span is the extra future time whose probability we want. Conditional probability divides the chance of surviving all the way to s plus t by the chance of having reached s. The later event is contained inside the earlier condition, so this ratio is exact. Insert the exponential survival probabilities. The numerator contains e to the minus lambda times s plus t. The denominator contains e to the minus lambda s. The factors involving s cancel. What remains is e to the minus lambda t, exactly the probability of waiting at least t from a fresh start. The elapsed wait has disappeared from the answer. These two curves make the result visible. On the left is survival for an additional time starting now. On the right is survival for the same additional time after we have already waited s. The curves are identical. Why does this feel absurd? We often imagine an arrival that is already on its way, a bus following a timetable, or a component accumulating age. Those mechanisms carry information from the past. A constant-rate Poisson process does not: future tiny intervals are independent of past ones. Memorylessness is therefore not a universal law of waiting. It is a sharp consequence of the assumptions we chose. If the rate changes with time, arrivals repel one another, or a schedule is operating, the remaining wait can depend strongly on how long we have already waited.

Put the model to work at a service desk. Customers arrive at an average rate of six per hour. We want the probability of exactly three arrivals in half an hour, and the average time from now until the next arrival. For the count, match the units first. Thirty minutes is one half hour, so the expected count is lambda T, equal to six times one half, which is three. Use the Poisson probability with k equal to three and mu equal to three. The result is e to the minus three times three cubed over three factorial, about zero point two two four. For waiting, use the exponential mean. One over six hour is one sixth of sixty minutes, so the mean wait is ten minutes. The count calculation and the waiting calculation use the same rate lambda from two different perspectives. Now suppose a detector is observed in twelve equal one-minute windows. These are the recorded click counts. Equal windows matter because the Poisson prediction compares counts that share the same expected value. The twelve counts add to twenty-four, so their empirical mean is exactly two clicks per window. This estimates lambda times one minute. Next measure their spread around two. Using the average squared deviation, the empirical variance is twenty-six over twelve, about two point one seven. It is not exactly two, because a finite random sample does not reproduce a population identity perfectly. The mean two point zero zero and variance two point one seven are close. That does not prove the Poisson model, but it passes this first check. With many more windows, we could simulate or calculate the variation expected in these estimates and judge whether the discrepancy is unusual. The comparison becomes diagnostic when it is repeated over substantial data. Variance near the mean is compatible with independent arrivals at a stable rate. Variance much larger than the mean suggests extra variability: customers arriving in groups, bursts of detector noise, or a rate that changes through the day. Variance much smaller than the mean suggests excessive regularity: arrivals may inhibit one another, or a detector may briefly stop registering after each click. Either pattern points back to an assumption that should be reconsidered. So the complete connection is this. Chopping time into rare independent trials produces Poisson counts in the limit. Asking for zero counts up to time t produces exponential waiting. Independence gives memorylessness, and the equal mean and variance of the counts gives the model an observable signature. The formulas are useful because their assumptions are clear. Use the same rate lambda to move between counts and waits, then return to data and ask whether independence, a stable rate, and equal mean and variance are actually visible. A probability model earns its place by surviving that return.