🎉 75% of content is free forever — Unlock Premium from $10/mo →
CW
Search courses…
💼 Servicesℹ️ About✉️ ContactView Pricing Plansfrom $10

Sampling Distribution of the Mean — Foundation of Inference

Foundations of StatisticsSampling Distributions🟢 Free Lesson

Advertisement

Sampling Distribution of the Mean — Foundation of Inference

Foundations of Statistics

Why Sample Means Behave Predictably

The sampling distribution of the mean explains why repeated samples give consistent results and underpins all confidence intervals and hypothesis tests. Understanding this concept is the key to mastering statistical inference.

  • Clinical Trials — Determining whether drug effects are real or due to sampling variation
  • Polling — Estimating margin of error for survey results
  • Manufacturing — Quality control through process monitoring and control charts

The sampling distribution transforms individual randomness into collective predictability.


Core Concepts

The sampling distribution of the mean describes how the sample mean Xˉ\bar{X} varies across all possible samples of size nn.

DfSampling Distribution of the Mean

The sampling distribution of Xˉ\bar{X} is the probability distribution of the statistic Xˉ=1ni=1nXi\bar{X} = \frac{1}{n}\sum_{i=1}^{n}X_i computed over all possible samples of size nn from a population. It is the theoretical basis for confidence intervals and hypothesis tests.

Mean and Standard Error

E[Xˉ]=μ,SE(Xˉ)=σnE[\bar{X}] = \mu, \quad \text{SE}(\bar{X}) = \frac{\sigma}{\sqrt{n}}

Here,

  • μ\mu=Population mean
  • σ\sigma=Population standard deviation
  • nn=Sample size
  • σ/n\sigma/\sqrt{n}=Standard error of the mean

Key Insight

The standard error decreases as n\sqrt{n} increases — larger samples give more precise estimates of the mean.


Central Limit Theorem

ThCentral Limit Theorem (CLT)

For any population with mean μ\mu and finite variance σ2\sigma^2, let X1,X2,,XnX_1, X_2, \ldots, X_n be i.i.d. random variables. Then as nn \to \infty:

Xˉμσ/ndN(0,1)\frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \xrightarrow{d} N(0, 1)

Equivalently, XˉN(μ,σ2n)\bar{X} \approx N\left(\mu, \frac{\sigma^2}{n}\right) for large nn, regardless of the population distribution shape.

Proof Sketch (Lindeberg–Lévy CLT)

Step 1. Assume E[Xi]=μE[X_i] = \mu and Var(Xi)=σ2<\text{Var}(X_i) = \sigma^2 < \infty. Define Zi=(Xiμ)/σZ_i = (X_i - \mu)/\sigma, so E[Zi]=0E[Z_i] = 0 and Var(Zi)=1\text{Var}(Z_i) = 1.

Step 2. The moment generating function of Zˉ=1nZi\bar{Z} = \frac{1}{n}\sum Z_i is MZˉ(t)=[MZ(t/n)]nM_{\bar{Z}}(t) = \left[M_Z(t/n)\right]^n.

Step 3. Taylor-expand: MZ(s)=1+s2/2+o(s2)M_Z(s) = 1 + s^2/2 + o(s^2) as s0s \to 0. Thus MZˉ(t)=[1+t22n2+o(1/n2)]net2/2M_{\bar{Z}}(t) = \left[1 + \frac{t^2}{2n^2} + o(1/n^2)\right]^n \to e^{t^2/2}.

Step 4. Since et2/2e^{t^2/2} is the MGF of N(0,1)N(0,1), by Lévy's continuity theorem, nZˉdN(0,1)\sqrt{n}\,\bar{Z} \xrightarrow{d} N(0,1).

Rule of Thumb

The CLT approximation is generally valid when n30n \geq 30. For highly skewed or heavy-tailed populations, larger nn may be needed. The Berry–Esseen theorem quantifies the rate: Fn(x)Φ(x)Cρσ3n|F_n(x) - \Phi(x)| \leq \frac{C\,\rho}{\sigma^3 \sqrt{n}} where ρ=E[Xμ3]\rho = E[|X-\mu|^3].


Formal Properties of Xˉ\bar{X}

ThUnbiasedness and Minimum Variance

The sample mean Xˉ\bar{X} is an unbiased estimator of μ\mu: E[Xˉ]=μE[\bar{X}] = \mu. Moreover, among all linear unbiased estimators, Xˉ\bar{X} has the minimum variance (Gauss–Markov theorem for the i.i.d. case).

Proof Sketch

Unbiasedness: E[Xˉ]=E[1nXi]=1nE[Xi]=nμn=μE[\bar{X}] = E\left[\frac{1}{n}\sum X_i\right] = \frac{1}{n}\sum E[X_i] = \frac{n\mu}{n} = \mu.

Variance: Var(Xˉ)=1n2Var(Xi)=nσ2n2=σ2n\text{Var}(\bar{X}) = \frac{1}{n^2}\sum \text{Var}(X_i) = \frac{n\sigma^2}{n^2} = \frac{\sigma^2}{n} by independence. Any other linear combination aiXi\sum a_i X_i with ai=1\sum a_i = 1 has variance σ2ai2σ2/n\sigma^2 \sum a_i^2 \geq \sigma^2/n by Cauchy–Schwarz, with equality iff all ai=1/na_i = 1/n.


Worked Example

Suppose the heights of adult males in a city are normally distributed with μ=175\mu = 175 cm and σ=8\sigma = 8 cm. A researcher samples n=64n = 64 men.

Step 1. The sampling distribution of Xˉ\bar{X} is exactly:

XˉN(175,8264)=N(175,1)\bar{X} \sim N\left(175, \frac{8^2}{64}\right) = N(175, 1)

Step 2. The standard error is SE(Xˉ)=8/64=1\text{SE}(\bar{X}) = 8/\sqrt{64} = 1 cm.

Step 3. Probability the sample mean exceeds 177 cm:

P(Xˉ>177)=P(Z>1771751)=P(Z>2)=0.0228P(\bar{X} > 177) = P\left(Z > \frac{177 - 175}{1}\right) = P(Z > 2) = 0.0228

Step 4. Even though individual heights have σ=8\sigma = 8, the sample mean of 64 observations has SE=1\text{SE} = 1. The sampling distribution is 8 times narrower than the population distribution — a direct consequence of averaging.


Key Takeaways

Summary: Sampling Distribution of the Mean

  • Describes variability of Xˉ\bar{X} across all samples of size nn
  • Mean: E[Xˉ]=μE[\bar{X}] = \mu, Standard Error: SE=σ/n\text{SE} = \sigma/\sqrt{n}
  • CLT: XˉN(μ,σ2/n)\bar{X} \approx N(\mu, \sigma^2/n) for large nn (typically n30n \geq 30)
  • Standard error decreases with n\sqrt{n} — larger samples are more precise
  • Xˉ\bar{X} is the UMVUE (uniformly minimum variance unbiased estimator) of μ\mu under normality
  • Foundation for all confidence intervals and hypothesis tests about the mean

Premium Content

Sampling Distribution of the Mean — Foundation of Inference

Unlock this lesson and 900+ advanced tutorials with a Premium plan.

🎯End-to-end Projects
💼Interview Prep
📜Certificates
🤝Community Access

Already a member? Log in

Need Expert Statistics Help?

Get personalized tutoring, project support, or professional consulting.

Advertisement