Chi-Square Distribution — Sum of Squared Normals

Foundations of Statistics

The Foundation of Variance-Based Inference

The chi-square distribution arises from summing squared normal variables, making it essential for testing variances and independence. Its applications span from quality control to genetics, wherever squared deviations matter.

Genetics — Testing Hardy-Weinberg equilibrium in population studies
Manufacturing — Quality control through variance testing and goodness-of-fit
Market Research — Analyzing survey response patterns against expected distributions

The chi-square distribution connects normal theory to categorical data analysis.

Core Concepts

The chi-square distribution arises as the sum of squared independent standard normal random variables. It is fundamental to tests of variance and independence.

DfChi-Square Distribution

If $Z_1, Z_2, \ldots, Z_\nu$ are independent standard normals, then $Q = Z_1^2 + Z_2^2 + \cdots + Z_\nu^2$ follows a chi-square distribution with $\nu$ degrees of freedom, written $Q \sim \chi^2_\nu$ .

PDF of Chi-Square Distribution

f(x) = \frac{x^{\nu/2 - 1} e^{-x/2}}{2^{\nu/2} \Gamma(\nu/2)}, \quad x > 0

Here,

$\nu$ =Degrees of freedom
$\Gamma$ =Gamma function

Special Cases

$\chi^2_1$ = square of a single standard normal $Z^2$
$\chi^2_2$ = exponential distribution with rate $1/2$ (i.e., $2\chi^2_2 \sim \text{Exp}(1/2)$ )
When $\nu$ is large, $\chi^2_\nu \approx N(\nu, 2\nu)$ by the CLT

Interactive Visualization

Chi-Square Distribution — Interactive Explorer

Mean (μ) = 4.0000Var = 8.0000σ = 2.8284

Effect of Degrees of Freedom on Chi-Square Distribution

Mean (μ) = 4.0000Var = 8.0000σ = 2.8284

Mean, Variance, and Higher Moments

Chi-Square Mean and Variance

E[\chi^2_\nu] = \nu, \quad \text{Var}(\chi^2_\nu) = 2\nu

Here,

$\nu$ =Degrees of freedom

Proof

Mean: Since $E[Z_i^2] = \text{Var}(Z_i) + (E[Z_i])^2 = 1 + 0 = 1$ , we have $E[Q] = \sum_{i=1}^\nu E[Z_i^2] = \nu$ .

Variance: Since $Z_i^2$ are independent and $\text{Var}(Z_i^2) = E[Z_i^4] - (E[Z_i^2])^2 = 3 - 1 = 2$ (using the fourth moment of the normal):

\text{Var}(Q) = \sum_{i=1}^\nu \text{Var}(Z_i^2) = 2\nu

The skewness is $\gamma_1 = \sqrt{8/\nu}$ and excess kurtosis is $\gamma_2 = 12/\nu$ , both decreasing to 0 as $\nu \to \infty$ .

Relationship to Normal and Gamma

Connections

$\chi^2_\nu = \text{Gamma}(\alpha = \nu/2, \beta = 1/2)$ where $\beta$ is the rate parameter
$\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}$ for normal samples (used in variance tests)
Sum of independent chi-squares: if $Q_i \sim \chi^2_{\nu_i}$ independently, then $\sum Q_i \sim \chi^2_{\sum \nu_i}$

Derivation: Sample Variance and Chi-Square

ThDistribution of Sample Variance

If $X_1, \ldots, X_n \overset{\text{i.i.d.}}{\sim} N(\mu, \sigma^2)$ , then $\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}$ .

Proof Sketch

Step 1. Standardize: $Z_i = (X_i - \mu)/\sigma \sim N(0,1)$ , so $\sum Z_i^2 \sim \chi^2_n$ .

Step 2. Rewrite using the identity $\sum Z_i^2 = \sqrt{n}\,\bar{Z}^2 + \sum(Z_i - \bar{Z})^2$ .

Step 3. By Fisher's lemma, $\bar{Z}$ and the deviations $(Z_i - \bar{Z})$ are independent.

Step 4. $\sqrt{n}\,\bar{Z} \sim N(0,1)$ , so $\sqrt{n}\,\bar{Z}^2 \sim \chi^2_1$ . Therefore $\sum(Z_i - \bar{Z})^2 = \sum Z_i^2 - \sqrt{n}\,\bar{Z}^2 \sim \chi^2_{n-1}$ by the reproductive property.

Step 5. Since $s^2 = \frac{\sigma^2}{n-1}\sum(Z_i - \bar{Z})^2$ , we get $\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}$ .

Worked Example

A quality engineer tests whether a filling machine has variance $\sigma^2 = 4$ mL $^2$ . From $n = 25$ bottles, $s^2 = 6.3$ mL $^2$ .

Step 1. Compute the chi-square test statistic:

Q = \frac{(n-1)s^2}{\sigma_0^2} = \frac{24 \times 6.3}{4} = 37.8

Step 2. Under $H_0$ , $Q \sim \chi^2_{24}$ with $E[Q] = 24$ and $\text{Var}(Q) = 48$ , so $\text{SD}(Q) = 6.93$ .

Step 3. The observed value 37.8 is $(37.8 - 24)/6.93 = 1.99$ standard deviations above the mean.

Step 4. For a two-sided test at $\alpha = 0.05$ , the critical values are $\chi^2_{0.975, 24} = 12.401$ and $\chi^2_{0.025, 24} = 39.364$ . Since $37.8 < 39.364$ , we fail to reject $H_0$ .

Step 5. The chi-square distribution is asymmetric, so the test is inherently two-sided in a different way than z/t tests. The p-value is $2 \times \min(P(Q \leq 37.8), P(Q > 37.8))$ .

Asymmetry Matters

Unlike the normal distribution, the chi-square is right-skewed. The two-sided rejection region is not symmetric about $\nu$ . For $\nu = 24$ , the lower critical value is 12.4 but the upper is 39.4 — far from symmetric about 24.

Normal Approximation

ThWilson–Hilferty Transformation

For large $\nu$ , the chi-square distribution can be approximated by a normal via:

\left(\frac{\chi^2_\nu}{\nu}\right)^{1/3} \approx N\left(1 - \frac{2}{9\nu},\; \frac{2}{9\nu}\right)

This approximation is remarkably accurate even for $\nu$ as small as 5.

Key Takeaways

Summary: Chi-Square Distribution

Sum of $\nu$ squared independent standard normals: $\chi^2_\nu$
Mean: $\nu$ , Variance: $2\nu$ , Skewness: $\sqrt{8/\nu}$
Always positive and right-skewed; approaches normal for large $\nu$
Used in tests of variance: $(n-1)s^2/\sigma^2 \sim \chi^2_{n-1}$
Foundation for chi-square tests of independence and goodness-of-fit
Reproductive property: sum of independent chi-squares is chi-square

Chi-Square Distribution — Sum of Squared Normals

Chi-Square Distribution — Sum of Squared Normals

The Foundation of Variance-Based Inference

Core Concepts

DfChi-Square Distribution

PDF of Chi-Square Distribution

Interactive Visualization

Mean, Variance, and Higher Moments

Chi-Square Mean and Variance

Relationship to Normal and Gamma

Derivation: Sample Variance and Chi-Square

ThDistribution of Sample Variance

Worked Example

Normal Approximation

ThWilson–Hilferty Transformation

Key Takeaways

Summary: Chi-Square Distribution

Premium Content

Need Expert Statistics Help?