Normal Distribution — The Bell Curve and Its Properties

Foundations of Statistics

The Universal Language of Randomness

The normal distribution is the cornerstone of statistical theory and practice, appearing everywhere from natural phenomena to financial markets. Its mathematical elegance makes complex probability calculations tractable and enables powerful inferential techniques.

Quality Control — Manufacturing processes use normal distributions to set tolerance limits and detect defects
Finance — Asset returns and risk models rely on normal distribution assumptions for portfolio optimization
Social Sciences — Test scores, heights, and measurement errors follow approximately normal distributions

Understanding the bell curve unlocks the door to nearly all of classical statistics.

Why the Normal Distribution is Central

The normal (Gaussian) distribution is the most important probability distribution in all of statistics and the natural sciences. Three fundamental reasons account for its centrality:

The Central Limit Theorem guarantees that sums and averages of many independent random variables converge to a normal distribution, regardless of the underlying distribution.
Maximum entropy among all distributions with fixed mean and variance — it is the "least informative" assumption.
Mathematical tractability — closed-form expressions exist for its moments, moment-generating function, and convolutions.

Definition and Probability Density Function

DfNormal Distribution

A continuous random variable $X$ is said to have a normal distribution with mean $\mu$ and variance $\sigma^2$ , written $X \sim \mathcal{N}(\mu, \sigma^2)$ , if its probability density function is:

f(x) = \frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right), \quad x \in \mathbb{R}

Parameters of the Normal Distribution

f(x) = \frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)

Here,

$\mu$ =Location parameter — controls the center of the distribution
$\sigma$ =Scale parameter — controls the spread (σ > 0)
$\sigma^2$ =Variance — the second central moment
$\exp(\cdot)$ =Natural exponential function

Fundamental Properties

ThProperties of the Normal Distribution

Symmetry: $f(\mu + x) = f(\mu - x)$ for all $x$ . The distribution is symmetric about $\mu$ .
Unimodal: The single mode occurs at $x = \mu$ .
Inflection points: The density changes curvature at $x = \mu \pm \sigma$ .
Total probability: $\int_{-\infty}^{\infty} f(x)\,dx = 1$ .
Mean = Median = Mode: All three measures of central tendency coincide at $\mu$ .
The $\pm k\sigma$ rule: $P(\mu - k\sigma \leq X \leq \mu + k\sigma)$ depends only on $k$ .

The Normalizing Constant

The factor $\frac{1}{\sigma\sqrt{2\pi}}$ ensures the total area under the curve equals 1. It arises from the Gaussian integral:

\int_{-\infty}^{\infty} e^{-t^2/2}\,dt = \sqrt{2\pi}

This identity is fundamental to probability theory and connects to the Gamma function: $\Gamma(1/2) = \sqrt{\pi}$ .

The Standard Normal Distribution

DfStandard Normal Distribution

The standard normal is the special case $Z \sim \mathcal{N}(0, 1)$ , with PDF:

\phi(z) = \frac{1}{\sqrt{2\pi}} e^{-z^2/2}

Any normal random variable can be standardized via the transformation $Z = \frac{X - \mu}{\sigma}$ .

Standardization (Z-score transformation)

Z = \frac{X - \mu}{\sigma} \sim \mathcal{N}(0, 1)

Here,

$Z$ =Standard normal random variable
$X$ =Original normal random variable with X ~ N(μ, σ²)
$\mu$ =Mean of X
$\sigma$ =Standard deviation of X

Why Standardization Matters

Standardization converts any normal distribution to the standard normal, enabling the use of a single z-table (cumulative probability table) for all probability calculations. This is the foundation of all normal-based inference.

Interactive Visualization

Normal Distribution — Interactive Explorer

Mean (μ) = 0.0000Var = 1.0000σ = 1.0000

How to Use This Visualization

The interactive visualization above shows the normal distribution PDF. The shaded region represents the 95% probability area (±1.96σ). You can adjust the parameters μ (mean) and σ (standard deviation) to see how they affect the shape of the distribution. The vertical lines show the mean, median, and mode (all equal for the normal distribution).

Cumulative Distribution Function

The CDF of the standard normal has no closed-form expression:

Standard Normal CDF

\Phi(z) = P(Z \leq z) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^{z} e^{-t^2/2}\,dt

Here,

$\Phi(z)$ =CDF of the standard normal
$z$ =z-score

Key values from the standard normal table:

$z$	$\Phi(z)$	Interpretation
0	0.5000	50% of area is below the mean
1	0.8413	84.13% below $\mu + \sigma$
1.645	0.9500	95% below $\mu + 1.645\sigma$ (one-sided)
1.960	0.9750	97.50% below $\mu + 1.96\sigma$
2	0.9772	97.72% below $\mu + 2\sigma$
2.576	0.9950	99.50% below $\mu + 2.576\sigma$
3	0.9987	99.87% below $\mu + 3\sigma$

Standard Normal CDF — P(Z ≤ z)

The Empirical Rule (68-95-99.7)

ThEmpirical Rule

For $X \sim \mathcal{N}(\mu, \sigma^2)$ :

P(\mu - \sigma \leq X \leq \mu + \sigma) = 2\Phi(1) - 1 \approx 0.6827

P(\mu - 2\sigma \leq X \leq \mu + 2\sigma) = 2\Phi(2) - 1 \approx 0.9545

P(\mu - 3\sigma \leq X \leq \mu + 3\sigma) = 2\Phi(3) - 1 \approx 0.9973

This is the foundation of the $3\sigma$ rule: for normally distributed data, 99.7% of observations lie within 3 standard deviations of the mean. Observations beyond this range are potential outliers.

Comparing Normal Distributions

Effect of Standard Deviation on Normal Distribution

Mean (μ) = 0.0000Var = 1.0000σ = 1.0000

Understanding the Spread Parameter

As σ increases, the distribution becomes wider and shorter (more spread out). The area under each curve is still 1, but the probability is distributed over a larger range. This visualization shows why σ controls the "width" of the bell curve.

Moment-Generating Function

Moment-Generating Function of Normal Distribution

M_X(t) = E[e^{tX}] = \exp\left(\mu t + \frac{\sigma^2 t^2}{2}\right)

Here,

$M_X(t)$ =Moment-generating function
$\mu$ =Mean
$\sigma^2$ =Variance
$t$ =Real parameter (must exist)

Why the MGF is Powerful

The MGF uniquely determines the distribution. If two random variables have the same MGF (in a neighborhood of 0), they have the same distribution. The moments are recovered via $E[X^k] = M_X^{(k)}(0)$ — the $k$ -th derivative evaluated at $t=0$ .

Reproductive Property

ThLinear Combinations of Normals

If $X_1 \sim \mathcal{N}(\mu_1, \sigma_1^2)$ and $X_2 \sim \mathcal{N}(\mu_2, \sigma_2^2)$ are independent, then:

aX_1 + bX_2 \sim \mathcal{N}(a\mu_1 + b\mu_2,\; a^2\sigma_1^2 + b^2\sigma_2^2)

More generally, if $X_i \sim \mathcal{N}(\mu_i, \sigma_i^2)$ are independent, then:

\sum_{i=1}^n a_i X_i \sim \mathcal{N}\left(\sum a_i \mu_i, \; \sum a_i^2 \sigma_i^2\right)

This property is why the normal distribution is so pervasive — sums of normal random variables are always normal, making it closed under linear combinations.

Normal Approximation to the Binomial

Normal Approximation to Binomial

X \sim \text{Bin}(n, p) \;\approx\; Y \sim \mathcal{N}(np, \, np(1-p))

Here,

$n$ =Number of trials
$p$ =Probability of success
$np$ =Mean of the binomial
$np(1-p)$ =Variance of the binomial

The approximation improves as $n$ increases. A standard rule of thumb: apply when $np \geq 10$ and $n(1-p) \geq 10$ . A continuity correction ( $\pm 0.5$ ) improves accuracy for finite $n$ .

Python Implementation

Example: Working with Normal Distribution

import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

# Create normal distribution
mu, sigma = 0, 1
normal_dist = stats.norm(loc=mu, scale=sigma)

# Calculate statistics
mean = normal_dist.mean()
var = normal_dist.var()
std = normal_dist.std()

print(f"Mean: {mean:.4f}")
print(f"Variance: {var:.4f}")
print(f"Standard Deviation: {std:.4f}")

# Generate random samples
np.random.seed(42)
samples = normal_dist.rvs(size=10000)

# Plot histogram vs theoretical PDF
plt.figure(figsize=(10, 6))
plt.hist(samples, bins=50, density=True, alpha=0.7, label='Samples')
x = np.linspace(-4, 4, 1000)
plt.plot(x, normal_dist.pdf(x), 'r-', lw=2, label='Theoretical PDF')
plt.title('Normal Distribution (μ=0, σ=1)')
plt.xlabel('x')
plt.ylabel('Density')
plt.legend()
plt.grid(True, alpha=0.3)
plt.show()

# Calculate probabilities
print(f"\nP(-1 ≤ X ≤ 1) = {normal_dist.cdf(1) - normal_dist.cdf(-1):.4f}")
print(f"P(-2 ≤ X ≤ 2) = {normal_dist.cdf(2) - normal_dist.cdf(-2):.4f}")
print(f"P(-3 ≤ X ≤ 3) = {normal_dist.cdf(3) - normal_dist.cdf(-3):.4f}")

# Percentiles
print(f"\n95th percentile: {normal_dist.ppf(0.95):.4f}")
print(f"99th percentile: {normal_dist.ppf(0.99):.4f}")

Key Takeaways

Summary: Normal Distribution

Symmetric, bell-shaped density centered at $\mu$ with spread $\sigma$
Standardization: $Z = (X - \mu)/\sigma \sim \mathcal{N}(0,1)$ — converts any normal to the standard normal
Empirical rule: approximately 68%, 95%, 99.7% within 1, 2, 3 standard deviations
Reproductive property: linear combinations of independent normals are normal
Central Limit Theorem: sums/means of many i.i.d. random variables converge to normal
MGF uniquely determines the distribution: $M_X(t) = \exp(\mu t + \sigma^2 t^2/2)$
Foundation for inference: z-tests, t-tests, ANOVA, and regression all rely on normality

Normal Distribution — The Bell Curve and Its Properties

Normal Distribution — The Bell Curve and Its Properties

The Universal Language of Randomness

Why the Normal Distribution is Central

Definition and Probability Density Function

DfNormal Distribution

Parameters of the Normal Distribution

Fundamental Properties

ThProperties of the Normal Distribution

The Standard Normal Distribution

DfStandard Normal Distribution

Standardization (Z-score transformation)

Interactive Visualization

Cumulative Distribution Function

Standard Normal CDF

The Empirical Rule (68-95-99.7)

ThEmpirical Rule

Comparing Normal Distributions

Moment-Generating Function

Moment-Generating Function of Normal Distribution

Reproductive Property

ThLinear Combinations of Normals

Normal Approximation to the Binomial

Normal Approximation to Binomial

Python Implementation

Example: Working with Normal Distribution

Key Takeaways

Summary: Normal Distribution

Premium Content

Need Expert Statistics Help?