Time Series Analysis

Why It Matters

Time series appear everywhere: stock prices, sensor data, user traffic, weather records, GDP growth. Unlike cross-sectional data, time series have temporal structure — trends, seasonality, and autocorrelation — that requires specialized methods. Understanding these patterns is crucial for forecasting, anomaly detection, and understanding dynamic systems. Ignoring temporal structure leads to invalid inference and poor predictions.

Overview

A time series decomposes into four components: trend (long-term direction), seasonality (fixed-period patterns like daily or yearly cycles), cyclical (irregular long-term fluctuations), and noise (random variation). Most time series methods require stationarity — constant mean and variance over time. The ADF test checks for a unit root (non-stationarity). The ARIMA framework applies differencing to achieve stationarity, autoregressive terms to capture dependence on past values, and moving-average terms to capture dependence on past errors. ACF/PACF plots guide model order selection. Once fitted, the model extrapolates future values, but confidence intervals widen with forecast horizon.

Key Concepts

ARIMA(p, d, q) Model

\phi(B)(1-B)^d X_t = \theta(B)\epsilon_t

Here,

$\phi(B)$ =Autoregressive polynomial: $1 - \phi_1 B - \cdots - \phi_p B^p$
$(1-B)^d$ =Differencing operator of order d
$\theta(B)$ =Moving average polynomial: $1 + \theta_1 B + \cdots + \theta_q B^q$
$\epsilon_t$ =White noise error: $\epsilon_t \sim N(0, \sigma^2)$

Backshift Operator

B X_t = X_{t-1}, \quad B^k X_t = X_{t-k}

Here,

$B$ =Backshift (lag) operator

AR(1) Model

X_t = \phi X_{t-1} + \epsilon_t

Here,

$\phi$ =Autoregressive coefficient; $|\phi| < 1$ for stationarity

MA(1) Model

X_t = \epsilon_t + \theta \epsilon_{t-1}

Here,

$\theta$ =Moving average coefficient

Exponential Smoothing (Simple)

\hat{X}_{t+1} = \alpha X_t + (1-\alpha)\hat{X}_t

Here,

$\alpha$ =Smoothing parameter ($0 < \alpha < 1$)

Time Series Components

Component	Description	Example
Trend	Long-term direction	Increasing user base
Seasonality	Fixed-period pattern	Daily traffic peaks at noon
Cyclical	Irregular long-term swings	Business cycles (years)
Noise	Random variation	Weather fluctuations

ACF/PACF Pattern Guide

Model	ACF Pattern	PACF Pattern
AR(p)	Tails off (exponential/sinusoidal)	Cuts off after lag p
MA(q)	Cuts off after lag q	Tails off
ARMA(p,q)	Tails off	Tails off

Quick Example

Interpreting ADF Test

ADF test on a time series: statistic = -1.2, p-value = 0.85.

Cannot reject the null of a unit root. The series is non-stationary. Apply differencing ( $d \geq 1$ in ARIMA) until the ADF p-value < 0.05.

After first differencing: ADF statistic = -4.5, p-value = 0.001. Now stationary — use ARIMA with $d = 1$ .

ARIMA Order Selection

ACF cuts off after lag 1 -> MA(1). PACF cuts off after lag 2 -> AR(2). ACF tails off, PACF cuts off after lag 2 -> AR(2). Use AIC to choose between competing models — lower AIC = better fit penalized for complexity.

Key Takeaways

Summary: Time Series Analysis

Four Components: Trend, seasonality, cyclical, noise. Identifying these guides model selection.
Stationarity: Required for most models. Constant mean and variance over time. Test with ADF test.
ARIMA: AR (autoregressive lags) + I (differencing for stationarity) + MA (moving average of errors). Select order via ACF/PACF or AIC.
Backshift Operator: $B X_t = X_{t-1}$ simplifies notation. $(1-B)^d$ removes trends of order $d$ .
Forecasting: Extrapolate with forecast(steps=10), but confidence intervals widen with horizon — shorter forecasts are more reliable.
Workflow: Plot series -> test stationarity -> difference if needed -> identify ARIMA order -> fit -> validate residuals -> forecast.
Seasonal Extension: SARIMA adds seasonal AR, I, and MA terms for periodic patterns.

Deep Dive

For detailed explanations, worked examples, and Python implementations, explore the dedicated statistics lessons:

Stationarity

Time Series Stationarity — ADF test, KPSS test, differencing strategies, and achieving stationarity

Model Identification

ACF and PACF — Autocorrelation and partial autocorrelation functions for identifying ARIMA order with examples

ARIMA Models

ARIMA Models — Fitting, diagnostics, forecasting, order selection, and Python implementation

Seasonal Models

Seasonal Decomposition — STL decomposition, seasonal patterns, trend estimation, and decomposition methods

Smoothing Methods

Exponential Smoothing — Simple, double, and triple (Holt-Winters) exponential smoothing for forecasting

Causality

Granger Causality — Testing whether one time series helps predict another

Time Series Analysis