The Gaussian Integral (Normalization Constant)Open in Colab

The Gaussian Integral (Normalization Constant)

The Gaussian PDF is p(x)=1σ2π e−(x−μ)2/2σ2p(x) = \frac{1}{\sigma\sqrt{2\pi}}\,e^{-(x-\mu)^2/2\sigma^2}. The constant 1σ2π\frac{1}{\sigma\sqrt{2\pi}} is not given. It is derived by demanding that the total area equal 1. Computing it requires one beautiful trick: squaring the integral and switching to polar coordinates.

Evaluating the standard Gaussian integral

Call the total area II:

I=∫−∞∞e−x2/2 dx.I = \int_{-\infty}^\infty e^{-x^2/2}\, dx.

There is no elementary antiderivative for e−x2/2e^{-x^2/2}. The trick is to compute I2I^2 instead.

Step 1: square the integral. Write it as a product of two copies with different dummy variables:

I2=(∫−∞∞e−x2/2 dx) ⁣(∫−∞∞e−y2/2 dy)=∫−∞∞ ⁣ ⁣∫−∞∞e−(x2+y2)/2 dx dy.I^2 = \left(\int_{-\infty}^\infty e^{-x^2/2}\, dx\right)\!\left(\int_{-\infty}^\infty e^{-y^2/2}\, dy\right) = \int_{-\infty}^\infty\!\!\int_{-\infty}^\infty e^{-(x^2+y^2)/2}\, dx\, dy.

Step 2: switch to polar coordinates (x=rcos⁡θx = r\cos\theta, y=rsin⁡θy = r\sin\theta, area element dx dy=r dr dθdx\, dy = r\, dr\, d\theta):

I2=∫02π ⁣ ⁣∫0∞e−r2/2 r dr dθ.I^2 = \int_0^{2\pi}\!\!\int_0^\infty e^{-r^2/2}\, r\, dr\, d\theta.

The angle integral gives 2π2\pi. The radial integral becomes elementary under u=r2/2u = r^2/2, du=r drdu = r\, dr:

∫0∞e−r2/2 r dr=∫0∞e−u du=[−e−u]0∞=0−(−1)=1.\int_0^\infty e^{-r^2/2}\, r\, dr = \int_0^\infty e^{-u}\, du = \Big[-e^{-u}\Big]_0^\infty = 0 - (-1) = 1.

Step 3: conclude.

I2=2π⋅1=2π⟹I=2π.I^2 = 2\pi \cdot 1 = 2\pi \quad\Longrightarrow\quad I = \sqrt{2\pi}.

Scaling the constant to general μ\mu and σ\sigma

For N(μ,σ2)\mathcal{N}(\mu, \sigma^2), substitute z=(x−μ)/σz = (x - \mu)/\sigma, dx=σ dzdx = \sigma\, dz:

∫−∞∞e−(x−μ)2/2σ2 dx=σ∫−∞∞e−z2/2 dz=σ2π.\int_{-\infty}^\infty e^{-(x-\mu)^2/2\sigma^2}\, dx = \sigma \int_{-\infty}^\infty e^{-z^2/2}\, dz = \sigma\sqrt{2\pi}.

Dividing by σ2π\sigma\sqrt{2\pi} forces the area to 1 — that is the normalization constant.

Deriving E[X]=μ\mathbb{E}[X] = \mu

Use the same substitution z=(x−μ)/σz = (x - \mu)/\sigma, so x=μ+σzx = \mu + \sigma z and dx=σ dzdx = \sigma\, dz. Let ϕ(z)=12πe−z2/2\phi(z) = \frac{1}{\sqrt{2\pi}}e^{-z^2/2} be the standard normal density:

E[X]=∫−∞∞x 1σ2πe−(x−μ)2/2σ2 dx=∫−∞∞(μ+σz) ϕ(z) dz.\mathbb{E}[X] = \int_{-\infty}^\infty x\, \frac{1}{\sigma\sqrt{2\pi}} e^{-(x-\mu)^2/2\sigma^2}\, dx = \int_{-\infty}^\infty (\mu + \sigma z)\, \phi(z)\, dz.

Split:

=μ∫−∞∞ϕ(z) dz⏟=1+σ∫−∞∞z ϕ(z) dz⏟=0=μ.= \mu \underbrace{\int_{-\infty}^\infty \phi(z)\, dz}_{= 1} + \sigma \underbrace{\int_{-\infty}^\infty z\,\phi(z)\, dz}_{= 0} = \mu.

The first integral is 1 (normalization). The second is 0 because z ϕ(z)z\,\phi(z) is an odd function (g(−z)=−g(z)g(-z) = -g(z)) integrated over the symmetric interval (−∞,∞)(-\infty, \infty).

Deriving Var(X)=σ2\text{Var}(X) = \sigma^2

Var(X)=E[(X−μ)2]=σ2∫−∞∞z2ϕ(z) dz\text{Var}(X) = \mathbb{E}[(X-\mu)^2] = \sigma^2 \int_{-\infty}^\infty z^2 \phi(z)\, dz. We need the integral to equal 1.

Integration by parts. First, note ϕ′(z)=ddz ⁣[12πe−z2/2]=12πe−z2/2⋅(−z)=−z ϕ(z)\phi'(z) = \frac{d}{dz}\!\left[\tfrac{1}{\sqrt{2\pi}}e^{-z^2/2}\right] = \tfrac{1}{\sqrt{2\pi}}e^{-z^2/2}\cdot(-z) = -z\,\phi(z). This means z ϕ(z) dz=−dϕz\,\phi(z)\, dz = -d\phi. Setting u=zu = z, dv=z ϕ(z) dzdv = z\,\phi(z)\, dz: Set u=zu = z, dv=z ϕ(z) dzdv = z\,\phi(z)\, dz, giving du=dzdu = dz and v=−ϕ(z)v = -\phi(z):

∫−∞∞z2ϕ(z) dz=[−z ϕ(z)]−∞∞+∫−∞∞ϕ(z) dz.\int_{-\infty}^\infty z^2 \phi(z)\, dz = \Big[-z\,\phi(z)\Big]_{-\infty}^{\infty} + \int_{-\infty}^\infty \phi(z)\, dz.

The boundary term lim⁡z→±∞z ϕ(z)=0\lim_{z \to \pm\infty} z\,\phi(z) = 0 because the Gaussian decays faster than zz grows. The remaining integral is 1. Therefore:

Var(X)=σ2⋅1=σ2.\text{Var}(X) = \sigma^2 \cdot 1 = \sigma^2.

Verifying all of this numerically

Python
import numpy as np
from scipy import integrate

# Verify I = sqrt(2*pi)
I_numeric, _ = integrate.quad(lambda x: np.exp(-x**2 / 2), -np.inf, np.inf)
print(f"Numeric I  = {I_numeric:.10f}")
print(f"sqrt(2π)   = {np.sqrt(2 * np.pi):.10f}")

# Monte Carlo check: verify E[X] and Var[X] for N(mu, sigma^2)
rng = np.random.default_rng(0)
mu, sigma = 3.0, 2.0
samples = rng.normal(mu, sigma, size=500_000)
print(f"\nN({mu}, {sigma**2}) via Monte Carlo (N=500k):")
print(f"  E[X]  = {samples.mean():.5f}  (true: {mu})")
print(f"  Var[X]= {samples.var():.5f}  (true: {sigma**2:.1f})")

# Gaussian integral approximation error vs number of Monte Carlo samples
errors = []
Ns = [100, 1000, 10_000, 100_000]
for N in Ns:
    x = rng.normal(0, 1, N)
    # Estimate I via importance sampling: I = E[e^{x^2/2} * e^{-x^2/2} * sqrt(2pi)] = sqrt(2pi)
    # Simpler: just verify normalization numerically
    _, err = integrate.quad(lambda t: np.exp(-t**2 / 2), -10, 10)
    errors.append(abs(integrate.quad(lambda t: np.exp(-t**2/2), -N/100, N/100)[0] - np.sqrt(2*np.pi)))

print("\nTruncation error of integral over [-N/100, N/100]:")
for N, e in zip(Ns, errors):
    print(f"  N={N:>6}: error={e:.2e}")

Related concepts

Concept neighbourhood

How this concept connects — click any node to jump there, or explore the whole concept map.

RelatedReferenced by
The Gaussian Integral (Normalizat…Probability Distributions…MLE for the GaussianBeta-Bernoulli Conjugate …Probability Distributions