Laplace's Integration Method

Suppose we want to compute an integral of the form $$ \int_a^b f(x) \exp(-\lambda g(x))dx, $$ as \(\lambda\rightarrow\infty\), where

The core concept here is that, when \(\lambda\) is very, very big, the only place in \((a,b)\) where the integral doesn't vanish is inside some (small) neighborhood of the critical point \(x^*\).

We can Taylor expand both \(f\) and \(g\) in a \(\delta\)-neighborhood about the point \(x=x^*\), which gives $$ \begin{equation} \begin{aligned} I(\lambda) &= \int_a^b f(x) \exp(-\lambda g(x))dx \\ &\approx \int_{x^* - \delta}^{x^* + \delta} \left(f(x^*) + (x-x^*)f^\prime(x^*) + \dots\right)\exp{\left( -\lambda(g(x^*) + (x-x^*)g^\prime(x^*) + \dots) \right)}dx \\ \end{aligned} \end{equation} $$

Remember that \(x^*\) is a local minimum, so we have \(g^\prime(x^*) = 0\) and \(g^{\prime\prime}(x^*)>0\). We may now approximate \(I\) as $$ \begin{equation} \begin{aligned} I(\lambda) &\approx \exp{\left(-\lambda g(x^*)\right)}\int_{x^* - \delta}^{x^* + \delta} \left(f(x^*) + (x-x^*)f^\prime(x^*) + \dots\right)\exp{\left( -\lambda(x-x^*)g^\prime(x^*) + \dots \right)}dx \\ &= \exp{\left(-\lambda g(x^*)\right)}\left[f(x^*)\int_{x^* - \delta}^{x^* + \delta}\exp{\left(-\frac{\lambda}{2}g^{\prime\prime}(x^*)(x-x^*)^2\right)dx} + h^\prime(x^*)\int_{x^* - \delta}^{x^* + \delta}(x-x^*)\exp{\left(-\frac{\lambda}{2}g^{\prime\prime}(x^*)(x-x^*)^2\right)dx} \right] \end{aligned} \end{equation} $$

Let's take a look at the took integrals on the right-hand side. First we have $$ \int_{x^* - \delta}^{x^* + \delta}\exp{\left(-\frac{\lambda}{2}g^{\prime\prime}(x^*)(x-x^*)^2\right)dx} $$ If we squint, we can recognize this as a Gaussian integral: $$ \begin{equation} \begin{aligned} \int_{x^* - \delta}^{x^* + \delta}\exp{\left(-\frac{\lambda}{2}g^{\prime\prime}(x^*)(x-x^*)^2\right)dx} &= \int_{x^* - \delta}^{x^* + \delta}\exp{\left(-\frac{1}{2}\left[\frac{x-x^*}{\sqrt{\frac{1}{\lambda g^{\prime\prime}(x^*)}}}\right]^2\right)dx} \\ &\approx \int_{-\infty}^{\infty}\exp{\left(-\frac{1}{2}\left[\frac{x-x^*}{\sqrt{\frac{1}{\lambda g^{\prime\prime}(x^*)}}}\right]^2\right)dx} \\ &= \sqrt{\frac{2\pi}{\lambda g^{\prime\prime}(x^*)}} \end{aligned} \end{equation} $$

The second integral now makes more sense in light of the first one: $$ \begin{equation} \begin{aligned} \int_{x^* - \delta}^{x^* + \delta}(x-x^*)\exp{\left(-\frac{\lambda}{2}g^{\prime\prime}(x^*)(x-x^*)^2\right)dx} &\approx \int_{-\infty}^{\infty}(x-x^*)\exp{\left(-\frac{\lambda}{2}g^{\prime\prime}(x^*)(x-x^*)^2\right)dx} \\ &= O(\text{E}\left[x-x^*\right]) \\ &= 0 \end{aligned} \end{equation} $$

The change of limits of integration can seem a bit shady and deserves discussion. The approximation $$ \int_{x^* - \delta}^{x^* + \delta}\exp{\left(-\frac{1}{2}\left[\frac{x-x^*}{\sqrt{\frac{1}{\lambda g^{\prime\prime}(x^*)}}}\right]^2\right)dx} \\ \approx \int_{-\infty}^{\infty}\exp{\left(-\frac{1}{2}\left[\frac{x-x^*}{\sqrt{\frac{1}{\lambda g^{\prime\prime}(x^*)}}}\right]^2\right)dx} \\ $$ relies on the bulk of the contributions of this integral lying inside the \(\delta\)-neighborhood around the critical point \(x^*\). As the parameter \(\lambda\rightarrow\infty\), we should see that the integrand becomes even more concentrated around the critical point, increasing the accuracy of the approximation.

Now we can put all this together as $$ \begin{equation} \begin{aligned} I(\lambda) \rightarrow \left({\frac{2\pi}{\lambda g^{\prime\prime}(x^*)}}\right)^{1/2}\,\,\frac{f(x^*)}{e^{\lambda g(x^*)}} \end{aligned} \end{equation} $$ as \(\lambda\rightarrow\infty\).
Q.E.D.

This remarkable result is apparently due to Laplace in 1774, introduced to approximate integrals arising in connection with statistical inference!

References