Standard Error — Precision of Sample Statistics

Foundations of Statistics

Measuring the Reliability of Estimates

The standard error quantifies how precisely a sample statistic estimates the population parameter, forming the foundation for confidence intervals and hypothesis tests. It decreases with sample size, enabling better estimates with more data.

Medical Research — Assessing precision of treatment effect estimates in clinical trials
Polling — Reporting margins of error for survey estimates
Economics — Evaluating precision of economic indicators and forecasts

The standard error tells you how much trust to place in your estimates.

Core Concepts

The standard error (SE) measures the precision of a sample statistic — it quantifies how much the statistic would vary across repeated samples.

DfStandard Error

The standard error of a statistic $\hat{\theta}$ is the standard deviation of its sampling distribution: $\text{SE}(\hat{\theta}) = \sqrt{\text{Var}(\hat{\theta})}$ . For the sample mean, $\text{SE}(\bar{X}) = \sigma/\sqrt{n}$ (when $\sigma$ is known) or $\text{SE}(\bar{X}) = s/\sqrt{n}$ (when estimated from data).

Standard Error of the Mean

\text{SE}(\bar{X}) = \frac{\sigma}{\sqrt{n}} \approx \frac{s}{\sqrt{n}}

Here,

$\sigma$ =Population standard deviation
$s$ =Sample standard deviation
$n$ =Sample size

SE vs SD

Standard deviation (SD) describes spread of individual data points: $\text{SD}(X) = \sigma$
Standard error (SE) describes precision of the sample mean as an estimate: $\text{SE}(\bar{X}) = \sigma/\sqrt{n}$
SE is always smaller than SD (by factor $1/\sqrt{n}$ )

Formal Derivation

ThVariance of the Sample Mean

If $X_1, \ldots, X_n$ are i.i.d. with variance $\sigma^2$ , then $\text{Var}(\bar{X}) = \sigma^2/n$ .

Proof

\text{Var}(\bar{X}) = \text{Var}\left(\frac{1}{n}\sum_{i=1}^n X_i\right) = \frac{1}{n^2}\sum_{i=1}^n \text{Var}(X_i) = \frac{n\sigma^2}{n^2} = \frac{\sigma^2}{n}

The crucial step is that the variance of a sum of independent random variables equals the sum of their variances. Taking the square root: $\text{SE}(\bar{X}) = \sigma/\sqrt{n}$ .

SE for Proportions

Standard Error of a Proportion

\text{SE}(\hat{p}) = \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}

Here,

$\hat{p}$ =Sample proportion
$n$ =Sample size

Derivation for Proportions

Let $X_i \sim \text{Bernoulli}(p)$ , so $\hat{p} = \frac{1}{n}\sum X_i$ . Since $\text{Var}(X_i) = p(1-p)$ :

\text{Var}(\hat{p}) = \frac{p(1-p)}{n}, \qquad \text{SE}(\hat{p}) = \sqrt{\frac{p(1-p)}{n}}

Replacing $p$ with $\hat{p}$ gives the estimated SE. The maximum of $p(1-p)$ occurs at $p = 0.5$ , giving the conservative bound $\text{SE} \leq \frac{1}{2\sqrt{n}}$ .

How Sample Size Affects SE

ThSample Size and Precision

To reduce the standard error by a factor $k$ , the sample size must increase by a factor $k^2$ .

Proof

If $\text{SE}_1 = \sigma/\sqrt{n_1}$ and we want $\text{SE}_2 = \text{SE}_1/k$ , then:

\frac{\sigma}{\sqrt{n_2}} = \frac{1}{k}\cdot\frac{\sigma}{\sqrt{n_1}} \implies \sqrt{n_2} = k\sqrt{n_1} \implies n_2 = k^2 n_1

Examples: To halve the SE, quadruple $n$ . To reduce SE by factor 10, need 100× the sample. This square-root law is fundamental to experimental design.

Worked Example

A pharmaceutical company measures blood pressure reduction. From prior studies, $\sigma = 12$ mmHg.

$n$	$\text{SE} = 12/\sqrt{n}$	95% CI half-width
25	$12/5 = 2.40$	$\pm 4.70$
100	$12/10 = 1.20$	$\pm 2.35$
400	$12/20 = 0.60$	$\pm 1.18$
1600	$12/40 = 0.30$	$\pm 0.59$

Key observation: Going from $n=100$ to $n=400$ (4× the sample) halves the SE from 1.20 to 0.60. The cost of data collection must be weighed against this diminishing return.

Practical Implication

The $\sqrt{n}$ law creates a fundamental trade-off: each doubling of precision requires quadrupling the budget. This is why clinical trials use formal power analysis before collecting data.

Estimated Standard Error

Replacing $\sigma$ with $s$

When $\sigma$ is unknown (almost always), we estimate it with $s = \sqrt{\frac{1}{n-1}\sum(X_i - \bar{X})^2}$ . The estimator $s^2$ is unbiased for $\sigma^2$ (by Bessel's correction), but $s$ is a biased estimator of $\sigma$ (by Jensen's inequality, since $\sqrt{\cdot}$ is concave). For large $n$ , the bias is negligible.

Key Takeaways

Summary: Standard Error

SE measures precision of a sample statistic (how much it varies across samples)
SE of mean: $\sigma/\sqrt{n}$ (known $\sigma$ ) or $s/\sqrt{n}$ (estimated)
SE of proportion: $\sqrt{\hat{p}(1-\hat{p})/n}$
SE decreases with $\sqrt{n}$ : to halve SE, quadruple $n$ (square-root law)
SE is the denominator of z-scores, t-scores, and confidence intervals
Always distinguish between $\sigma$ (population parameter) and $s$ (sample estimate)

Standard Error — Precision of Sample Statistics

Standard Error — Precision of Sample Statistics

Measuring the Reliability of Estimates

Core Concepts

DfStandard Error

Standard Error of the Mean

Formal Derivation

ThVariance of the Sample Mean

SE for Proportions

Standard Error of a Proportion

How Sample Size Affects SE

ThSample Size and Precision

Worked Example

Estimated Standard Error

Key Takeaways

Summary: Standard Error

Premium Content

Need Expert Statistics Help?