# The exponential distribution and the Poisson process

This post is a continuation of the previous post on the exponential distribution. The previous post discusses the basic mathematical properties of the exponential distribution including the memoryless property. This post looks at the exponential distribution from another angle by focusing on the intimate relation with the Poisson process.

_______________________________________________________________________________________________

The Poisson Process

A previous post shows that a sub family of the gamma distribution that includes the exponential distribution is derived from a Poisson process. This post gives another discussion on the Poisson process to draw out the intimate connection between the exponential distribution and the Poisson process.

Suppose a type of random events occur at the rate of $\lambda$ events in a time interval of length 1. As the random events occur, we wish to count the occurrences. Starting at time 0, let $N(t)$ be the number of events that occur by time $t$. A counting process is the collection of all the random variables $N(t)$. That is, we are interested in the collection $\left\{N(t): t \ge 0 \right\}$. More specifically, we are interested in a counting process that satisfies the following three axioms:

• The probability of having more than one occurrence in a short time interval is essentially zero.
• The probability of the occurrence of a random event in a short time interval is proportional to the length of the time interval and not on where the time interval is located. More specifically, the probability of the occurrence of the random event in a short interval of length $h$ is approximately $\lambda \ h$.
• The numbers of random events occurring in non-overlapping time intervals are independent.

Any counting process that satisfies the above three axioms is called a Poisson process with the rate parameter $\lambda$. Of special interest are the counting random variables $N(t)$, which is the number of random events that occur in the interval $[0,t]$ and $N(s+t)-N(t)$, which is the number of events that occur in the interval $[s,s+t]$. There are also continuous variables that are of interest. For example, the time until the occurrence of the first event, denoted by $S_1$, and in general, the time until the occurrence of the $n$th event, denoted by $S_n$. We are also interested in the interarrival times, i.e. the time between the occurrences of two consecutive events. These are notated by $T_1, T_2, T_3,\cdots$ where $T_n$ is the time between the occurrence of the $(n-1)$st event and the occurrence of the $n$th event. Of course, $S_n=T_1+T_2+\cdots+T_n$.

_______________________________________________________________________________________________

Some Basic Results

Given a Poisson process $\left\{N(t): t \ge 0 \right\}$ with rate parameter $\lambda>0$, we discuss the following basic results:

Counting Variables

• The discrete random variable $N(t)$ is a Poisson random variable with mean $\lambda \ t$.
• The discrete random variable $N(s+t)-N(s)$ is a Poisson random variable with mean $\lambda \ t$.

Waiting Time Variables

• The interarrival times, $T_n$, where $n=1,2,3,\cdots$, are independent and identically distributed exponential random variables with rate parameter $\lambda$ (or mean $\frac{1}{\lambda}$).
• The waiting time until the $n$th event has a gamma distribution with shape parameter $n$ and rate parameter $\lambda$.

The result that $N(t)$ is a Poisson random variable is a consequence of the fact that the Poisson distribution is the limit of the binomial distribution. To see this, let’s say we have a Poisson process with rate $\lambda>0$. Note thar $\lambda$ is the rate of occurrence of the event per unit time interval. Then subdivide the interval $[0,t]$ into $n$ subintervals of equal length. When $n$ is sufficiently large, we can assume that there can be only at most one event occurring in a subinterval (using the first two axioms in the Poisson process). Each subinterval is then like a Bermoulli trial (either 0 events or 1 event occurring in the subinterval). The probability of having exactly one event occurring in a subinterval is approximately $\lambda (\frac{t}{n})=\frac{\lambda t}{n}$. By the third criterion in the Poisson process, the $n$ subintervals are independent Bernoulli trials. Thus the total number of events occurring in these $n$ subintervals is a Binomial random variable with $n$ trials and with probability of success in each trial being $\frac{\lambda t}{n}$. It can be shown mathematically that when $n \rightarrow \infty$, the binomial distributions converge to the Poisson distribution with mean $\lambda \ t$. This fact is shown here and here. It follows that $N(t)$ has a Poisson distribution with mean $\lambda \ t$.

To show that the increment $N(s+t)-N(s)$ is a Poisson distribution, we simply count the events in the Poisson process starting at time $s$. It is clear that the resulting counting process is also a Poisson process with rate $\lambda$. We can use the same subdivision argument to derive the fact that $N(s+t)-N(s)$ is a Poisson random variable with mean $\lambda \ t$. The subdividing is of course on the interval $[s,s+t]$.

Based on the preceding discussion, given a Poisson process with rate parameter $\lambda$, the number of occurrences of the random events in any interval of length $t$ has a Poisson distribution with mean $\lambda \ t$. Thus in a Poisson process, the number of events that occur in any interval of the same length has the same distribution. Any counting process that satisfies this property is said to possess stationary increments. On the other hand, any counting process that satisfies the third criteria in the Poisson process (the numbers of occurrences of events in disjoint intervals are independent) is said to have independent increments. Thus a Poisson process possesses independent increments and stationary increments.

Here is an interesting observation as a result of the possession of independent increments and stationary increments in a Poisson process. By independent increments, the process from any point forward is independent of what had previously occurred. By stationary increments, from any point forward, the occurrences of events follow the same distribution as in the previous phase. Starting with a Poisson process, if we count the events from some point forward (calling the new point as time zero), the resulting counting process is probabilistically the same as the original process. The probabilistic behavior of the new process from some point on is not dependent on history. The probability statements we can make about the new process from some point on can be made using the same parameter as the original process. In other words, a Poisson process has no memory.

We now discuss the continuous random variables derived from a Poisson process. The time until the first change, $T_1$, has an exponential distribution with mean $\frac{1}{\lambda}$. To see this, for $T_1>t$ to happen, there must be no events occurring in the interval $[0,t]$. Thus, $P[T_1>t]$ is identical to $P[T_1>t]=P[N(t)=0]=e^{-\lambda t}$. This means that $T_1$ has an exponential distribution with rate $\lambda$. What about $T_2$ and $T_3$ and so on? After the first event had occurred, we can reset the counting process to count the events starting at time $T_1$. Then the time until the next occurrence is also an exponential random variable with rate $\lambda$. Consequently, all the interarrival times $T_n$ are exponential random variables with the same rate $\lambda$. Furthermore, by the discussion in the preceding paragraph, the exponential interarrival times $T_1, T_2, T_3,\cdots$ are independent.

As a consequence of the $T_n$ being independent exponential random variables, the waiting time until the $n$th change $S_n$ is a gamma random variable with shape parameter $n$ and rate parameter $\lambda$. Note that $S_n=T_1+\cdots+T_n$ and that independent sum of $n$ identical exponential distribution has a gamma distribution with parameters $n$ and $\lambda$, which is the identical exponential rate parameter.

The connection between exponential/gamma and the Poisson process provides an expression of the CDF and survival function for the gamma distribution when the shape parameter is an integer. Specifically, the following shows the survival function and CDF of the waiting time $S_n$ as well as the density.

$\displaystyle P(S_n>t)=\sum \limits_{k=0}^{n-1} \frac{e^{-\lambda t} \ (\lambda t)^k}{k!}$

$\displaystyle P(S_n \le t)=1-\sum \limits_{k=0}^{n-1} \frac{e^{-\lambda t} \ (\lambda t)^k}{k!}=\sum \limits_{k=n}^{\infty} \frac{e^{-\lambda t} \ (\lambda t)^k}{k!}$

$\displaystyle f_{S_n}(t)=\frac{1}{(n-1)!} \ \lambda^n \ t^{n-1} \ e^{-\lambda t}$

The key in establishing the survival $\displaystyle P(S_n>t)$ is that the waiting time $S_n$ is intimately related to $N(t)$, which has a Poisson distribution with mean $\lambda t$. For $S_n>t$ to happen, there can be at most $n$ events occurring prior to time $t$, i.e. $N(t) \le t$.

_______________________________________________________________________________________________

Reversing the Poisson Process

The preceding discussion shows that a Poisson process has independent exponential waiting times between any two consecutive events and gamma waiting time between any two events. Starting with a collection $\left\{N(t): t \ge 0 \right\}$ of Poisson counting random variables that satisfies the three axioms described above, it can be shown that the sequence of interarrival times are independent exponential random variables with the same rate parameter as in the given Poisson process. Interestingly, the process can also be reversed, i.e. given a sequence of independent and identically distributed exponential distributions, each with rate $\lambda$, a Poisson process can be generated.

To see this, let $T_1,T_2,T_3,\cdots$ be a sequence of independent and identically distributed exponential random variables with rate parameter $\lambda$. Mathematically, the $T_n$ are just independent and identically distributed exponential random variables. Now think of them as the interarrival times between consecutive events. So the first event occurs at time $S_1=T_1$ and the second event occurs at time $S_2=T_1+T_2$ and so on. In general, the $n$th event occurs at time $S_n=T_1+\cdots+T_n$. More specifically, the counting process is $\left\{N(t): t \ge 0 \right\}$ where $N(t)$ is defined below:

$N(t)=\text{max} \left\{n: S_n=T_1+\cdots+T_n \le t \right\}$

For $N(t)=n$ to happen, it must be true that $S_n=T_1+\cdots+T_n \le t$ and $S_{n+1}=T_1+\cdots+T_n+T_{n+1} > t$. The probability $P[N(t)=n]$ is then

\displaystyle \begin{aligned}P[N(t)=n]&=P[S_{n+1} > t \text{ and } S_n \le t]\\&=P[S_{n+1} > t \text{ and } (\text{ not } S_n > t)] \\&=P[S_{n+1} > t]-P[S_n > t] \\&=\sum \limits_{k=0}^{n} \frac{e^{-\lambda t} \ (\lambda t)^k}{k!}-\sum \limits_{k=0}^{n-1} \frac{e^{-\lambda t} \ (\lambda t)^k}{k!} \\&=\frac{e^{-\lambda t} \ (\lambda t)^n}{n!} \end{aligned}

The above derivation shows that the counting variable $N(t)$ is a Poisson random variable with mean $\lambda t$. The derivation uses the gamma survival function derived earlier. On the other hand, because of the memoryless property, $T_1-s,T_2-s,T_3-s,\cdots$ are also independent exponential random variables with the same rate $\lambda$. If the counting of events starts at a time $s$ rather than at time 0, the counting would be based on $T_k-s,T_{k+1}-s,T_{k+3}-s,\cdots$ for some $k$. By the same argument, $N(s+t)-N(s)$ would be Poisson with mean $\lambda t$.

We have just established that the resulting counting process from independent exponential interarrival times $T_n$ has stationary increments. The resulting counting process has independent increments too. This is because the interarrival times $T_n$ are independent and that the interarrival times $T_n$ are also memoryless.

From a mathematical point of view, a sequence of independent and identically distributed exponential random variables leads to a Poisson counting process. On the other hand, if random events occur in such a manner that the times between two consecutive events are independent and identically and exponentially distributed, then such a process is a Poisson process. This characterization gives another way work with Poisson processes.

_______________________________________________________________________________________________

Examples

Example 1
Suppose that the time until the next departure of a bus at a certain bus station is exponentially distributed with mean 10 minutes. Assume that the times in between consecutive departures at this bus station are independent.

• Tom and his friend Mike are to take a bus trip together. Tom arrives at the bus station at 12:00 PM and is the first one to arrive. Mike arrives at the bus stop at 12:30 PM. They will board the first bus to depart after the arrival of Mike. What is the the probability that zero buses depart from this bus station while Tom is waiting for Mike?
• Answer the same question for one bus, and two buses?
• What is the probability that there are at least three buses leaving the station while Tom is waiting?

Because the inter-departure times are independent and exponential with the same mean, the random events (bus departures) occur according to a Poisson process with rate $\frac{1}{10}$ per minute, or 1 bus per 10 minutes. The number of bus departures in a 30-minute period is a Poisson random variable with mean 3 (per 30 minutes). Let $N$ be the number of buses leaving the bus station between 12:00 PM and 12:30 PM. Thus the answers are:

$P[N=0]=e^{-3}=0.05$

$P[N=1]=e^{-3} \times 3=0.149$

$P[N=2]=e^{-3} \times \frac{3^2}{2}=0.224$

$P[N \ge 3]=1-P[n=0]-P[n=1]-P[n=2]=0.5768$ $\square$

Example 2
Taxi arrives at a certain street corner according to a Poisson process at the rate of two taxi for every 15 minutes. Suppose that you are waiting for a taxi at this street corner and you are third in line. Assume that the people waiting for taxi do not know each other and each one will have his own taxi. What is the probability that you will board a taxi within 30 minutes?

The number of arrivals of taxi in a 30-minute period has a Poisson distribution with a mean of 4 (per 30 minutes). If there are at least 3 taxi arriving, then you are fine. Let $N$ be the number of arrivals of taxi in a 30-minute period. The answer is

\displaystyle \begin{aligned}P[N \ge 3]&=1-P[N=0] -P[N=1]-P[N=2]\\&=1-e^{-4}-e^{-4} \times 4-e^{-4} \times \frac{4^2}{2} \\&=1-0.2381 \\&=0.7619 \end{aligned}

_______________________________________________________________________________________________

Remarks

In this post, we present a view of the exponential distribution through the view point of the Poisson process. Any counting process that satisfies the three axioms of a Poisson process has independent and exponential waiting time between any two consecutive events and gamma waiting time between any two events. On the other hand, given a sequence of independent and identically distributed exponential interarrival times, a Poisson process can be derived. The memoryless property of the exponential distribution plays a central role in the interplay between Poisson and exponential.

_______________________________________________________________________________________________
$\copyright \ 2016 - \text{Dan Ma}$