This post focuses on the mathematical properties of the exponential distribution. Since the exponential distribution is a special case of the gamma distribution, the starting point of the discussion is on the properties that are inherited from the gamma distribution. The discussion then switches to other intrinsic properties of the exponential distribution, e.g. the memoryless property. The next post discusses the intimate relation with the Poisson process. Additional topics on exponential distribution are discussed in this post.
The exponential distribution is highly mathematically tractable. For the exponential distribution with mean (or rate parameter ), the density function is . Thus for the exponential distribution, many distributional items have expression in closed form. The exponential distribution can certainly be introduced by performing calculation using the density function. For the sake of completeness, the distribution is introduced as a special case of the gamma distribution.
The Gamma Perspective
The exponential distribution is a special case of the gamma distribution. Recall that the gamma distribution has two parameters, the shape parameter and the rate parameter . Another parametrization uses the scale parameter , which is . For now we stick with the rate parameter because of the connection with the Poisson process discussed below. The exponential distribution is simply a gamma distribution when . Immediately the properties of the gamma distribution discussed in the previous post can be transferred here. Suppose that is a random variable that follows the exponential distribution with rate parameter , equivalently with mean . Immediately we know the following.
The above items are obtained by plugging into the results in the post on gamma distribution. It is clear that exponential distribution is very mathematically tractable. The CDF has a closed form. The moments have a closed form. As a result, it is possible to derive many more properties than the ones shown here.
The Memoryless Property
No discussion on the exponential distribution is complete without the mentioning of the memoryless property. Suppose a random variable has support on the interval . The random variable is said to have the memoryless property (or the “no memory” property) if
holds for all positive real numbers and . To get an appreciation of this property, let’s think of as the lifetime (in years) of some machine or some device. The statement in says that if the device has lived at least years, then the probability that the device will live at least more years is the same as the probability that a brand new device lives to age years. It is as if the device does not remember that it has already been in use for years. If the lifetime of a device satisfies this property, it does not matter if you buy an old one or a new one. Both old and new have the same probability of living an additional years. In other words, old device is as good as new.
Since the following is true,
the memoryless property is equivalent to the following:
The property says that the survival function of this distribution is a multiplicative function. The exponential distribution satisfies this property, i.e.
On the other hand, any continuous function that satisfies the multiplicative property must be an exponential function (see the argument at the end of the post). Thus there is only one continuous probability distribution that possesses the memoryless property. The memoryless property can also be stated in a different but equivalent way as follows:
The conditional random variable is distributed identically as the unconditional random variable
The statement can be rewritten . This means the conditional variable shares the same survival function as the original random variable , hence sharing the same cumulative distribution and the same density function and so on.
The memoryless property, anyone of the three statements, is a striking property. Once again, thinking of as the lifetime of a system or a device. The conditional random variable in statement is the remaining lifetime for a device at age . Statement says that it does not matter how old the device is, the remaining useful life is still governed by the same probability distribution for a brand new device, i.e. it is just as likely for an old device to live more years as for a new device, and that an old device is expected to live as long as a new device.
Let’s look a few examples.
Suppose that a bank has two tellers serving retailed customers that walk into the bank lobby. A queue is set up for customers to wait for one of the tellers. The time between the arrivals of two consecutive customers in the queue has an exponential distribution with mean 3 minutes. The time each teller spent with a customer has an exponential distribution with mean 5 minutes. Assume that the service times for the two tellers are independent. At 12:00 PM, both tellers are busy and a customer has just arrived in the queue.
- What is the probability that the next arrival in the queue will come before 12:05 PM? Between 12:05 and 12:10? After 12:10 PM?
- If no additional customers arrive before 12:05 PM, what is the probability that the next arrival will come within the next 5 minutes?
- If both tellers are busy serving customers at 1:00 PM, what is the probability that neither teller will finish the service before 1:05 PM? Before 1:10 PM?
Let be the waiting time between two consecutive customers and be the service time of a bank teller, both in minutes. The answers for Part 1 are
Part 2 involves the memoryless property. The answer is:
Part 3 also involves the memoryless property. It does not matter how long each server has spent with the customer prior to 1:00 PM, the remaining service time after 1:00 PM still has the same exponential service time distribution. The answers are:
Suppose that times between fatal auto accidents on a stretch of busy highway have an exponential distribution with a mean of 20 days. Suppose that an accident occurred on July 1. What is the probability that another fatal accident occurred in the same month? If the month of July were accident-free in this stretch of highway except for the accident on July 1, what is the probability that there will be another fatal accident in the following month (August)?
Let be the time in days from July 1 to the next fatal accident on this stretch of highway. Then is exponentially distributed with a mean of 20 days. The probability that another fatal accident will occur in the month of July is , which is
Note that the month of July has 31 days and the month of August has 31 days. If the month of July is accident-free except for the accident on July 1, then the probability that an accident occurs in August is:
In setting , the memoryless property is used in the above derivation . If the occurrence of fatal accidents is a random event and furthermore, if the time between two successive accidents is exponentially distributed, then there is no “memory” in the waiting of the next fatal accident. Having a full month of no accidents has no bearing on when the next fatal accident occurs.
Suppose that the amount of damage in an automobile accident follows an exponential distribution with mean 2000. An insurance coverage is available to cover such damages subject to a deductible of 1000. That is, if the damage amount is less than the deductible, the insurance pays nothing. If the damage amount is greater than the deductible, the policy pays the damage amount in excess of the deductible. Determine the mean, variance and standard deviation of the insurance payment per accident.
For clarity, the example is first discussed using and , where is the mean of the exponential damage and is the deductible. Let be the amount of the damage of an auto accident. Let be the amount paid by the insurance policy per accident. Then the following is the rule for determining the amount of payment.
With this payment rule, , and can be worked out based on the exponential random variable for the damage amount as follows:
Because the exponential distribution is mathematically very tractable, the mean and the variance are very doable. Indeed the above integrals are excellent exercise for working with exponential distribution. We would like to demonstrate a different approach. Because of the memoryless property, there is no need to calculate the above integrals.
The insurance payment is a mixture. Specifically it can be one of two possibilities. With probability , . With probability , . Because is an exponential random variable, is distributed identically as the original damage amount . Thus the mean of , , is the weighted average of and . Likewise is also a weighted average. The following shows how to calculate the first two moments of .
The second moment of the random variable is . The variance of the insurance payment is
In this example, and . We have:
Using the memoryless property and the fact that the insurance is a mixture requires less calculation. If the damage amount is not exponential, then we may have to resort to the direct calculation by doing the above integrals.
The Unique Distribution with the Memoryless Property
Now we show that exponential distribution is the only one with the memoryless property. First establish the fact that any right continuous function defined on satisfying the functional relation must be an exponential function. The statement that is a right continuous function means that if and for all , then .
Let be a right continuous function that is defined on such that it satisfies the functional relation . First, establish the following:
for any positive integers and .
Note that for any positive integer , . It follows that for any positive integer , . On the other hand, . To see this, note that . Raising both sides to gives the claim. Combining these two claims give the fact stated above. We have established the fact that for any positive rational number .
Next, we show that for any . To see this, let be a sequence of rational numbers converging to from the right. Then . By the above fact, for all . On the other hand, . The same sequence converges to both and . Thus . This means that is an exponential function with base . If the natural log constant is desired as a base, where .
Suppose that is memoryless. Let be the survival function of the random variable , i.e. . Then by property , . So the survival function satisfies the functional relation . The survival function is always right continuous. By the fact that any right continuous function satisfying the functional relation must be an exponential function, the survival function must be an exponential function. Thus is an exponential random variable. This establishes the fact that among the continuous distributions, the exponential distribution is the only one with the memoryless property.