The Gamma Distribution

The gamma function was developed by Euler as a generalization of the factorial function. It is defined by the integral $$ \Gamma(x) = \int_0^\infty t^{x-1} e^{-t} dt, $$ for real numbers \(x>0\). This is sometimes called Euler's integral of the second kind.
The gamma function can be further extended to cover the \(x<0\) case. See the book by Emil Artin [0] for more information.

This integral arose from the observation that $$ n! = \int_0^\infty t^n e^{-t} dt, $$ which can be shown using repeated application of integration by parts.

The gamma distribution is defined by $$ \text{Gamma}(x|\text{shape}=a,\text{rate}=b)=\frac{b^a}{\Gamma(a)}x^{a-1}e^{-bx}, $$ for \(x>0,\,a>0,\,b>0\).

Observe the relation between the integrand in the first equation and the gamma probability distribution function (PDF): $$ \frac{b^a}{\Gamma(a)}x^{a-1}e^{-bx} = \frac{b}{\Gamma(a)} (bx)^{a-1} e^{-bx} $$

From this we can find the first moment using the substitution \(t=bx\) like so $$ \text{E}\left[x\right] = \int_0^\infty \frac{b^a}{\Gamma(a)}x^{a}e^{-bx} dx = \frac{1}{b\Gamma(a)}\int_0^\infty t^{a} e^{-t} dt = \frac{a}{b\Gamma(a)} = a/b $$

The variance is given by $$ \text{Var}\left[x\right] = \text{E}\left[x^2\right] - \text{E}\left[x\right]^2. $$ The first term is found to be $$ \begin{equation} \begin{aligned} \text{E}\left[x^2\right] = \int_0^\infty \frac{b^a}{\Gamma(a)}x^{a+1}e^{-bx} dx &= \frac{b^2}{b^2}\text{E}\left[x^2\right] \\ &= \int_0^\infty \frac{b^{a+2}}{b^2 \Gamma(a)}x^{a+1}e^{-bx} dx \\ &= \frac{\Gamma(a+2)}{b^2\Gamma(a)} = \frac{a^2 + a}{b^2} \end{aligned} \end{equation} $$ and the second is \( \text{E}\left[x\right]^2 = {a^2}/{b^2}. \) Therefore $$ \text{Var}\left[x\right] = \frac{a}{b^2} $$

References