Various parametric continuous probability models have been presented and discussed in this blog. The number of parameters in these models ranges from one to two, and in a small number of cases three. They are all potential candidates for models of severity in insurance applications and in other actuarial applications. This post highlights these models. The list presented here is not exhaustive; it is only a brief catalog. There are other models that are also suitable for actuarial applications but not accounted for here. However, the list is a good place to begin. This post also serves a navigation device (the table shown below contains links to the blog posts).
Many of the models highlighted here are related to gamma distribution either directly or indirectly. So the catalog starts with the gamma distribution at the top and then branches out to the other related models. Mathematically, the gamma distribution is a two-parameter continuous distribution defined using the gamma function. The gamma sub family includes the exponential distribution, Erlang distribution and chi-squared distribution. These are distributions that are gamma distributions with certain restrictions on the one or both of the gamma parameters. Other distributions are obtained by raising a distribution to a power. Others are obtained by mixing distributions.
Here’s a listing of the models. Click on the links to find out more about the distributions.
|Gamma sub families|
|Independent sum of gamma|
|Raising to a power||Raising exponential to a positive power|
|Raising exponential to a power|
|Raising gamma to a power|
|Raising Pareto to a power|
|Burr sub families|
The above table categorizes the distributions according to how they are mathematically derived. For example, the gamma distribution is derived from the gamma function. The Pareto distribution is mathematically an exponential-gamma mixture. The Burr distribution is a transformed Pareto distribution, i.e. obtained by raising a Pareto distribution to a positive power. Even though these distributions can be defined simply by giving the PDF and CDF, knowing how their mathematical origins informs us of the specific mathematical properties of the distributions. Organizing according to the mathematical origin gives us a concise summary of the models.
Further Comments on the Table
From a mathematical standpoint, the gamma distribution is defined using the gamma function.
In this above integral, the argument is a positive number. The expression in the integrand is always positive. The area in between the curve and the x-axis is . When this expression is normalized, i.e. divided by , it becomes a density function.
The above function is defined over all positive . The integral of over all positive is 1. Thus is a density function. It only has one parameter, the , which is the shape parameter. Adding the scale parameter making it a two-parameter distribution. The result is called the gamma distribution. The following is the density function.
Both parameters and are positive real numbers. The first parameter is the shape parameter and is the scale parameter.
As mentioned above, many of the distributions listed in the above table is related to the gamma distribution. Some of the distributions are sub families of gamma. For example, when are positive integers, the resulting distributions are called Erlang distribution (important in queuing theory). When , the results are the exponential distributions. When and where is a positive integer, the results are the chi-squared distributions (the parameter is referred to the degrees of freedom). The chi-squared distribution plays an important role in statistics.
Taking independent sum of independent and identically distributed exponential random variables produces the Erlang distribution, a sub gamma family of distribution. Taking independent sum of exponential random variables, with pairwise distinct means, produces the hypoexponential distributions. On the other hand, the mixture of independent exponential random variables produces the hyperexponential distribution.
The Pareto distribution (Pareto Type II Lomax) is the mixture of exponential distributions with gamma mixing weights. Despite the connection with the gamma distribution, the Pareto distribution is a heavy tailed distribution. Thus the Pareto distribution is suitable for modeling extreme losses, e.g. in modeling rare but potentially catastrophic losses.
As mentioned earlier, raising a Pareto distribution to a positive power generates the Burr distribution. Restricting the parameters in a Burr distribution in a certain way will produces the paralogistic distribution. The table indicates the relationships in a concise way. For details, go into the blog posts to get more information.
Another informative way to categorize the distributions listed in the table is through looking at the tail weight. At first glance, all the distributions may look similar. For example, the distributions in the table are right skewed distributions. Upon closer look, some of the distributions put more weights (probabilities) on the larger values. Hence some of the models are more suitable for models of phenomena with significantly higher probabilities of large or extreme values.
When a distribution significantly puts more probabilities on larger values, the distribution is said to be a heavy tailed distribution (or said to have a larger tail weight). In general tail weight is a relative concept. For example, we say model A has a larger tail weight than model B (or model A has a heavier tail than model B). However, there are several ways to check for tail weight of a given distribution. Here are the four criteria.
|Tail Weight Measure||What to Look for|
|1||Existence of moments||The existence of more positive moments indicates a lighter tailed distribution.|
|2||Hazard rate function||An increasing hazard rate function indicates a lighter tailed distribution.|
|3||Mean excess loss function||An increasing mean excess loss function indicates a heavier tailed distribution.|
|4||Speed of decay of survival function||A survival function that decays rapidly to zero (as compared to another distribution) indicates a lighter tailed distribution.|
Existence of moments
For a positive real number , the moment is defined by the integral where is the density function of the distribution in question. If the distribution puts significantly more probabilities in the larger values in the right tail, this integral may not exist (may not converge) for some . Thus the existence of moments for all positive is an indication that the distribution is a light tailed distribution.
In the above table, the only distributions for which all positive moments exist are gamma (including all gamma sub families such as exponential), Weibull, lognormal, hyperexponential, hypoexponential and beta. Such distributions are considered light tailed distributions.
The existence of positive moments exists only up to a certain value of a positive integer is an indication that the distribution has a heavy right tail. All the other distributions in the table are considered heavy tailed distribution as compared to gamma, Weibull and lognormal. Consider a Pareto distribution with shape parameter and scale parameter . Note that the existence of the Pareto higher moments is capped by the shape parameter . If the Pareto distribution is to model a random loss, and if the mean is infinite (when ), the risk is uninsurable! On the other hand, when , the Pareto variance does not exist. This shows that for a heavy tailed distribution, the variance may not be a good measure of risk.
Hazard rate function
The hazard rate function of a random variable is defined as the ratio of the density function and the survival function.
The hazard rate is called the force of mortality in a life contingency context and can be interpreted as the rate that a person aged will die in the next instant. The hazard rate is called the failure rate in reliability theory and can be interpreted as the rate that a machine will fail at the next instant given that it has been functioning for units of time.
Another indication of heavy tail weight is that the distribution has a decreasing hazard rate function. On the other hand, a distribution with an increasing hazard rate function has a light tailed distribution. If the hazard rate function is decreasing (over time if the random variable is a time variable), then the population die off at a decreasing rate, hence a heavier tail for the distribution in question.
The Pareto distribution is a heavy tailed distribution since the hazard rate is (Pareto Type I) and (Pareto Type II Lomax). Both hazard rates are decreasing function.
The Weibull distribution is a flexible model in that when its shape parameter is , the Weibull hazard rate is decreasing and when , the hazard rate is increasing. When , Weibull is the exponential distribution, which has a constant hazard rate.
The point about decreasing hazard rate as an indication of a heavy tailed distribution has a connection with the fourth criterion. The idea is that a decreasing hazard rate means that the survival function decays to zero slowly. This point is due to the fact that the hazard rate function generates the survival function through the following.
Thus if the hazard rate function is decreasing in , then the survival function will decay more slowly to zero. To see this, let , which is called the cumulative hazard rate function. As indicated above, . If is decreasing in , has a lower rate of increase and consequently has a slower rate of decrease to zero.
In contrast, the exponential distribution has a constant hazard rate function, making it a medium tailed distribution. As explained above, any distribution having an increasing hazard rate function is a light tailed distribution.
The mean excess loss function
The mean excess loss is the conditional expectation . If the random variable represents insurance losses, mean excess loss is the expected loss in excess of a threshold conditional on the event that the threshold has been exceeded. Suppose that the threshold is an ordinary deductible that is part of an insurance coverage. Then is the expected payment made by the insurer in the event that the loss exceeds the deductible.
Whenever is an increasing function of the deductible , the loss is a heavy tailed distribution. If the mean excess loss function is a decreasing function of , then the loss is a lighter tailed distribution.
The Pareto distribution can also be classified as a heavy tailed distribution based on an increasing mean excess loss function. For a Pareto distribution (Type I) with shape parameter and scale parameter , the mean excess loss is , which is increasing. The mean excess loss for Pareto Type II Lomax is , which is also decreasing. They are both increasing functions of the deductible ! This means that the larger the deductible, the larger the expected claim if such a large loss occurs! If the underlying distribution for a random loss is Pareto, it is a catastrophic risk situation.
In general, an increasing mean excess loss function is an indication of a heavy tailed distribution. On the other hand, a decreasing mean excess loss function indicates a light tailed distribution. The exponential distribution has a constant mean excess loss function and is considered a medium tailed distribution.
Speed of decay of the survival function to zero
The survival function captures the probability of the tail of a distribution. If a distribution whose survival function decays slowly to zero (equivalently the cdf goes slowly to one), it is another indication that the distribution is heavy tailed. This point is touched on when discussing hazard rate function.
The following is a comparison of a Pareto Type II survival function and an exponential survival function. The Pareto survival function has parameters ( and ). The two survival functions are set to have the same 75th percentile, which is . The following table is a comparison of the two survival functions.
Note that at the large values, the Pareto right tails retain much more probabilities. This is also confirmed by the ratio of the two survival functions, with the ratio approaching infinity. Using an exponential distribution to model a Pareto random phenomenon would be a severe modeling error even though the exponential distribution may be a good model for describing the loss up to the 75th percentile (in the above comparison). It is the large right tail that is problematic (and catastrophic)!
Since the Pareto survival function and the exponential survival function have closed forms, We can also look at their ratio.
In the above ratio, the numerator has an exponential function with a positive quantity in the exponent, while the denominator has a polynomial in . This ratio goes to infinity as .
In general, whenever the ratio of two survival functions diverges to infinity, it is an indication that the distribution in the numerator of the ratio has a heavier tail. When the ratio goes to infinity, the survival function in the numerator is said to decay slowly to zero as compared to the denominator.
It is important to examine the tail behavior of a distribution when considering it as a candidate for a model. The four criteria discussed here provide a crucial way to classify parametric models according to the tail weight.
The processes of creating distributions from existing ones are an important topic in the study of probability models. Such processes expand the tool kit in the modeling process. Two examples: new distributions can be generated by taking independent sum of old ones or by mixing distributions (the result would be called a mixture). Another way to generate distributions is through raising a distribution to a power, which is the subject of this post. Start with a random variable (the base distribution). Then raising it to a constant generates a new distribution. In this post, the base distribution is the exponential distribution. The next post discusses transforming the gamma distribution.
Raising to a Power
Let be a random variable. Let be a nonzero constant. The new distribution is generated when is raised to the power of . Thus the random variable is the subject of the discussion in this post.
When , the distribution for is called transformed. When , the distribution for is called inverse. When and , the distribution for is called inverse transformed.
If the base distribution is exponential, then raising it to would produce a transformed exponential distribution for the case of , an inverse exponential distribution for the case of and an inverse transformed exponential distribution for the case with . If the base distribution is a gamma distribution, the three new distributions would be transformed gamma distribution, inverse gamma distribution and inverse transformed gamma distribution.
For the case of inverse transformed, we make the random variable by letting . The following summarizes the definition.
|Name of Distribution||Parameter||Random Variable|
The “transformed” distributions discussed here have two parameters, and ( for inverse exponential). The parameter is the shape parameter, which comes from the exponent . The scale parameter is added after raising the base distribution to a power.
Let be the random variable for the base exponential distribution. The following shows the information on the base exponential distribution.
Note that the above density function and CDF do not have the scale parameter. Once the base distribution is raised to a power, the scale parameter will be added to the newly created distribution.
The following gives the CDF and the density function of the transformed exponential distribution. The density function is obtained by taking the derivative of the CDF.
The following gives the CDF and the density function of the inverse exponential distribution.
The following gives the CDF and the density function of the inverse transformed exponential distribution.
The above derivation does not involve the scale parameter. Now it is added to the results.
|Inverse Transformed Exponential|
The transformed exponential distribution and the inverse transformed distribution have two parameters and . The inverse exponential distribution has only one parameter . The parameter is the scale parameter. The parameter , when there is one, is the shape parameter and it comes from the exponent when the exponential is raised to a power.
The above transformation starts with the exponential distribution with mean 1 (without the scale parameter) and the scale parameter is added back in at the end. We can also accomplish the same result by starting with an exponential variable with mean (scale parameter) . Then raising to , -1, and would generate the three distributions described in the above table. In this process, the scale parameter is baked into the base distribution. This makes it easier to obtain the moments of the “transformed” exponential distributions since the moments would be derived from exponential moments.
Connection with Weibull Distribution
Compare the density function for the transformed exponential distribution with the density of the Weibull distribution discussed here. Note that the two are identical. Thus raising an exponential distribution to where produces a Weibull distribution.
On the other hand, raising a Weillbull distribution to -1 produces an inverse Weillbull distribution (by definition). Let be the CDF of the base Weibull distribution where . Let’s find the CDF of . Then add the scale parameter.
(scale parameter added)
Note the the CDF of inverse Weibull distribution is identical to the one for inverse transformed exponential distribution. Thus transformed exponential distribution is identical to a Weibull distribution and inverse transformed exponential distribution is identical to an inverse Weibull distribution.
Since Weibull distribution is the same as transformed exponential distribution, the previous post on Weibull distribution can inform us on transformed exponential distribution. For example, assuming that the Weibull distribution (or transformed exponential) is a model for the time until death of a life, varying the shape parameter yields different mortality patterns. The following are two graphics from the proevious post.
Figure 1 shows the Weibull density functions for different values of the shape parameter (the scale parameter is fixed at 1). The curve for is the exponential density curve. It is clear that the green density curve () approaches the x-axis at a faster rate then the other two curves and thus has a lighter tail than the other two density curves. In general, the Weibull (transformed exponential) distribution with shape parameter has a lighter tail than the Weibull with shape parameter .
Figure 2 shows the failure rates for the Weibull (transformed exponential) distributions with the same three values of . Note that the failure rate for (blue) decreases over time and the failure rate for increases over time. The failure rate for is constant since it is the exponential distribution.
What is being displayed in Figure 2 describes a general pattern. When the shape parameter is , the failure rate decreases as time increases and the Weibull (transformed exponential) distribution is a model for infant mortality, or early-life failures. Hence these Weibull distributions have a thicker tail as shown in Figure 1.
When the shape parameter is , the failure rate increases as time increases and the Weibull (transformed exponential) distribution is a model for wear-out failures. As times go by, the lives are fatigued and “die off.” Hence these Weibull distributions have a lighter tail as shown in Figure 1.
When , the resulting Weibull (transformed exponential) distribution is exponential. The failure rate is constant and it is a model for random failures (failures that are independent of age).
Thus the transformed exponential family has a great deal of flexibility for modeling the failures of objects (machines, devices).
Moments and Other Distributional Quantities
The moments for the three “transformed” exponential distributions are based on the gamma function. The two inverse distributions have limited moments. Since the transformed exponential distribution is identical to Weibull, its moments are identical to that of the Weibull distribution. The moments of the “transformed” exponential distributions are where has an exponential distribution with mean (scale parameter) . See here for the information on exponential moments. The following shows the moments of the “transformed” exponential distributions.
|Name of Distribution||Moment|
|Inverse Transformed Exponential|
The function is the Gamma function. The transformed exponential moment exists for all . The moments are limited for the other two distributions. The first moment does not exist for the inverse exponential distribution. The inverse transformed exponential moment exist only for . Thus the inverse transformed exponential mean and variance exist only if the shape parameter is larger than 2.
The distributional quantities that are based on moments can be calculated (e.g. variance, skewness and kurtosis) when the moments are available. For all three "transformed" exponential distributions, percentiles are easily computed since the CDFs contain only one instance of the unknown . The following gives the mode of the three distributions.
|Name of Distribution||Mode|
|Transformed Exponential||for , else 0|
|Inverse Transformed Exponential|
Mathematically, the Weibull distribution has a simple definition. It is mathematically tractable. It is also a versatile model. The Weibull distribution is widely used in life data analysis, particularly in reliability engineering. In addition to analysis of fatigue data, the Weibull distribution can also be applied to other engineering problems, e.g. for modeling the so called weakest link model.This post gives an introduction to the Weibull distribution.
Defining the Weibull Distribution
A random variable is said to follow a Weibull distribution if has the following density function
where and are some fixed constants. The notation refers to the exponential function . As defined here, the Weibull distribution is a two-parameter distribution with being the shape parameter and being the scale parameter. The following is the cumulative distribution function (CDF).
Connection with the Exponential Distribution
The Weibull distribution can also arise naturally from the random sampling of an exponential random variable. A better way to view Weibull is through the lens of exponential. Taking an observation from an exponential distribution and raising it to a positive power will result in a Weibull observation. Specifically, the random variable has the same CDF as in if is an exponential random variable with mean . To see this, consider the following:
The idea of the Weibull distribution as a power of an exponential distribution simplifies certain calculation on the Weibull distribution. For example, a raw moment of the Weibull distribution is simply another raw moment of the exponential distribution. For an exponential random variable with mean , the raw moments are (details can be found here):
For the Weibull random variable with parameters and , i.e. where is the exponential random variable with mean , the following shows the mean and higher moments.
With the moments established, several other distributional quantities that are based on moments can also be established. The following shows the variance, skewness and kurtosis.
The notation here is the skewness. The notation is kurtosis. The excess kurtosis is . In some sources, the notation is to denote excess kurtosis. Of course and are the mean and variance, respectively.
Another calculation that is easily accessible for the Weibull distribution is that of the percentiles. It is easy to solve for in the CDF in . For example, to find the median, set the CDF equals to 0.5 and solves for , producing the following.
Another basic and important property to examine is the failure rate. The failure rate of a distribution is the ratio of the density function to its survival function. The following is the failure of the Weibull distribution.
See here for a discussion of the failure rate in conjunction with the exponential distribution. Suppose that the distribution in question is a lifetime distribution (time until termination or death). Then the failure rate can be interpreted as the rate of failure at the next instant given that the life has survived to time .
When the Parameters Vary
The discussion in the previous section might give the impression that all Weibull distributions (when the parameters vary) behave in the same way. We now look at examples showing that as (shape parameter) and/or (scale parameter) vary, the distribution will exhibit markedly different behavior. Note that when , the Weibull distribution is reduced to the exponential distribution.
The following diagram shows the PDFs of the Weibull distribution with , and where in all three cases.
Figure 1 shows the effect of the shape parameter taking on different values while keeping the scale parameter fixed. The effect is very pronounced on the skewness. All three density curves are right skewed. The PDF with (the blue curve) has a very strong right skew. The PDF with (the red curve) is exponential and has, by comparison, a much smaller skewness. The PDF with looks almost symmetric, though there is a clear and small right skew. This observation is borne out by the calculation. The skewness coefficients are (blue curve), (red curve) and (green curve).
Another clear effect of the shape parameter is the thickness of the tail (in this case the right tail). Figure 1 suggests that the PDF with (the blue curve) is higher than the other two density curves on the interval . As a result, the blue curve has more probability mass in the right tail. Thus the blue curve has a thicker tail comparing to the other two PDFs. For a numerical confirmation, the following table compares the probabilities in the right tail.
It is clear from the above table that the Weibull distribution with the blue curve assigns more probabilities to the higher values. The mean of the distribution for blue curve is 2. The right tail (over 5 times the mean) contains 4.23% of the probability mass (a small probability for sure but not negligible). The right tail (over 12.5 times the mean) still has a small probability of 0.00673 that cannot be totally ignored. On the other hand, the Weibull distribution for the green curve has a light tail. The mean of the distribution for the green curve is about 0.89. At (over 4.5 times of its mean), the tail probability is already negligible at . At (over 11 times of its mean), the tail probability is , practically zero.
Let’s compare the density curves in Figure 1 with their failure rates. The following figure shows the failure rates for these three Weibull distributions.
According to the definition in , the following shows the failure rate function for the three Weibull distributions.
The blue curve in Figure 1 () has a decreasing failure rate as shown in Figure 2. The failure rate function is constant for the case of (the exponential case). It is an increasing function for the case of . This comparison shows that the Weibull distribution is particularly useful for engineers and researchers who study the reliability of machines and devices. If the engineers believe that the failure rate is constant, then an exponential model is appropriate. If they believe that the failure rate increases with time or age, then a Weibull distribution with shape parameter is more appropriate. If the engineers believe that the failure rate decreases with time, then a Weibull distribution with shape parameter is more appropriate.
We now compare Weibull distributions with various values for (scale parameter) while keeping the shape parameter fixed. The following shows the density curves for the Weibull distributions with while keeping .
The effect of the scale parameter is to compress or stretch out the standard Weibull density curve, i.e. the Weibull distribution with . For example, the density function for is obtained by stretching out the density curve for the one with . The same overall shape is maintained while the density curve is being stretched or compressed. According to , the mean of the transformed distribution is increased (stretching) or decreased (compressing). For example, as is increased from 1 to 2, the mean has a two-fold increase. As the density curve is stretched, the resulting distribution is more spread out and the peak of the density curve decreases. The overall effect of changing the scale parameter is essentially a change in the scale in the x-axis.
The next example is a computational exercise.
The time until failure (in months) of a semiconductor device has a Weibull distribution with shape parameter and scale parameter .
- Give the density function and the survival function.
- Determine the probability that the device will last at least 500 hours.
- Determine the probability that the device will last at least 600 hours given that it has been running for over 500 hours.
- Find the mean and standard deviation of the time until failure.
- Determine the failure rate function of the Weibull time until failure.
To obtain the density function, the survival function and the failure rate, follow the relationships in , and .
Note that the Weibull failure rate is the ratio of the density function to the survival function. In this case, the failure is an increasing function of . Since is a time scale, then this is a model for machines that wear out over time (see the next section).
The probability that the device will last over 500 hours is . The unconditional probability that the device will last over 600 hours is The conditional probability that the device will last more than 600 hours given that it has lasted more 500 hours is the ratio .
To find the mean and variance, we need to evaluate the gamma function. Using Excel, we obtain the following two values of the gamma function (as shown here):
The mean and standard deviation of the time until failure are:
The Weibull Failure Rates
Looking at the failure rate function indicated in and looking at Figure 2, it is clear that when the shape parameter , the failure rate decreases with time (if the distribution is a model for the time until death of a life). When the shape parameter , the failure rate is constant. When the shape parameter , the failure rate increases with time. As a result, the Weibull family of distribution has a great deal of flexibility for modeling the failures of objects (machines, devices).
When the shape parameter , the failure rate decreases with time. Such a Weibull distribution is a model for infant mortality, or early-life failures. When the shape parameter (or near 1), the failure rate is constant or near constant. The resulting Weibull distribution (an exponential model) is a model for random failures (failures that are independent of age). When the shape parameter , the failure rate increases with time. The resulting Weibull distribution is a model for wear-out failures.
In some applications, it may be necessary to model each phase of a lifetime separately, e.g. the early phase with a Weibull distribution with , the useful phase with a Weibull distribution with close to 1 and the wear-out phase with a Weibull distribution with . The resulting failure rate curve resembles a bathtub curve. The following is an idealized bathtub curve.
The blue part of the bathtub curve is the early phase of the lifetime, which is characterized by decreasing failure rate. This is the early-life period in which the defective products die off and are taken out of the study. The next period is the useful-life period, the red part of the curve, in which the failures are random that are independent of age. In this phase, the failure rate is constant or near constant. The green part of the bathtub curve is characterized by increasing failure rates, which is the wear-out phase of the lifetime being studied.
Weakest Link Model
Another attractiveness of the Weibull model is that it can be used to model the so called the weakest link model. Consider a machine or device that has multiple components. Suppose that the device dies or fails when any one of the components fails. The lifetime of such a machine or device is the time to the first failure. Such a lifetime model is called the weakest link model. It can be shown that under these conditions a Weibull distribution is a good model for the distribution of the lifetime of such a machine or device.
If the time until failure of the individual components are indpendent and identically distributed Weibull random variables, then it follows that the minimum of of the Weibull random variables is also a Weibull random variable. To see this, let be independent and identically distributed Weibull random varaibles. Let and be the parameters for the common Weibull distribution. Let . The following gives the survival function of .
where and . This shows that has a Weibull distribution with shape parameter (as before) and scale parameter . Under the condition that the times to failure for the multiple components are indentically Weibull distributed, the lifetime of the device is also a Weibull model.
The Moment Generating Function
The post is concluded with a comment on the moment generating function for the Weibull distribution. Note that relationship indicates that all positive moments for the Weibull distribution exist. A natural question is on whether the moment generating function (MGF) exists. It turns out that MGF does not always exist for the Weibull distribution. The MGF exists for the Weibull distribution whenever the shape parameter . However the MGF cannot be expressed in terms of any familiar functions. Instead, the Weibull MGF can be expressed as a power series.
To see this, start with the power series for .
Since the positive moments exist for the Weibull distribution, the higher moments from are plugged into the power series. It can be shown that the last series converges when .
When , the power series does not converge. For a specific example, let . Then the term in the series is simplified to , when goes to infinity when . Thus the Weilbull distribution with the shape parameter is an example of a distribution where all the positive moments exist but the MGF does not exist.