The Gaussian Integral (Normalization Constant)
The Gaussian PDF is . The constant is not given. It is derived by demanding that the total area equal 1. Computing it requires one beautiful trick: squaring the integral and switching to polar coordinates.
Evaluating the standard Gaussian integral
Call the total area :
There is no elementary antiderivative for . The trick is to compute instead.
Step 1: square the integral. Write it as a product of two copies with different dummy variables:
Step 2: switch to polar coordinates (, , area element ):
The angle integral gives . The radial integral becomes elementary under , :
Step 3: conclude.
Scaling the constant to general and
For , substitute , :
Dividing by forces the area to 1 — that is the normalization constant.
Deriving
Use the same substitution , so and . Let be the standard normal density:
Split:
The first integral is 1 (normalization). The second is 0 because is an odd function () integrated over the symmetric interval .
Deriving
. We need the integral to equal 1.
Integration by parts. First, note . This means . Setting , : Set , , giving and :
The boundary term because the Gaussian decays faster than grows. The remaining integral is 1. Therefore:
Verifying all of this numerically
import numpy as np
from scipy import integrate
# Verify I = sqrt(2*pi)
I_numeric, _ = integrate.quad(lambda x: np.exp(-x**2 / 2), -np.inf, np.inf)
print(f"Numeric I = {I_numeric:.10f}")
print(f"sqrt(2π) = {np.sqrt(2 * np.pi):.10f}")
# Monte Carlo check: verify E[X] and Var[X] for N(mu, sigma^2)
rng = np.random.default_rng(0)
mu, sigma = 3.0, 2.0
samples = rng.normal(mu, sigma, size=500_000)
print(f"\nN({mu}, {sigma**2}) via Monte Carlo (N=500k):")
print(f" E[X] = {samples.mean():.5f} (true: {mu})")
print(f" Var[X]= {samples.var():.5f} (true: {sigma**2:.1f})")
# Gaussian integral approximation error vs number of Monte Carlo samples
errors = []
Ns = [100, 1000, 10_000, 100_000]
for N in Ns:
x = rng.normal(0, 1, N)
# Estimate I via importance sampling: I = E[e^{x^2/2} * e^{-x^2/2} * sqrt(2pi)] = sqrt(2pi)
# Simpler: just verify normalization numerically
_, err = integrate.quad(lambda t: np.exp(-t**2 / 2), -10, 10)
errors.append(abs(integrate.quad(lambda t: np.exp(-t**2/2), -N/100, N/100)[0] - np.sqrt(2*np.pi)))
print("\nTruncation error of integral over [-N/100, N/100]:")
for N, e in zip(Ns, errors):
print(f" N={N:>6}: error={e:.2e}")
Related concepts
- Probability Distributions: the full lesson introducing the Gaussian and its key properties
- MLE for the Gaussian: fitting and from data via maximum likelihood
- Beta-Bernoulli Conjugate Posterior: another exact derivation involving normalization constants