ARIMA Models — Complete Guide

Statistics

Combining Autoregression, Integration, and Moving Average

ARIMA models unify three powerful concepts — autoregressive dependence, differencing for stationarity, and moving average error correction — into a flexible framework for modeling and forecasting time series.

Financial Forecasting — Predict stock volatility and returns
Supply Chain — Forecast demand with seasonal patterns and trends
Energy — Model electricity consumption for grid management

The three parameters (p,d,q) encode the memory, trend, and noise structure of any time series.

ARIMA (Autoregressive Integrated Moving Average) models combine autoregression, differencing, and moving average components to model and forecast time series data.

DfARIMA(p,d,q) Model

An ARIMA(p,d,q) model is defined as:

ARIMA(p,d,q)

\phi(B)(1-B)^d Y_t = \theta(B)\varepsilon_t

Here,

$p$ =Order of the autoregressive (AR) part
$d$ =Degree of differencing
$q$ =Order of the moving average (MA) part
$B$ =Backshift operator: $BY_t = Y_{t-1}$
$\phi(B)$ =AR polynomial: $1 - \phi_1 B - \cdots - \phi_p B^p$
$\theta(B)$ =MA polynomial: $1 + \theta_1 B + \cdots + \theta_q B^q$
$\varepsilon_t$ =White noise error term

Component Models

AR(p) — Autoregressive

AR(p) Model

Y_t = c + \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \cdots + \phi_p Y_{t-p} + \varepsilon_t

Here,

$\phi_i$ =AR coefficient at lag i
$c$ =Constant (drift)

Stationarity Condition

An AR(p) process is stationary if all roots of $\phi(z) = 1 - \phi_1 z - \cdots - \phi_p z^p$ lie outside the unit circle.

MA(q) — Moving Average

MA(q) Model

Y_t = \mu + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \theta_2 \varepsilon_{t-2} + \cdots + \theta_q \varepsilon_{t-q}

Here,

$\theta_i$ =MA coefficient at lag i
$\mu$ =Mean of the process

Invertibility

An MA(q) process is invertible if all roots of $\theta(z) = 1 + \theta_1 z + \cdots + \theta_q z^q$ lie outside the unit circle.

ARMA(p,q)

ARMA(p,q) Model

Y_t = c + \sum_{i=1}^{p}\phi_i Y_{t-i} + \varepsilon_t + \sum_{j=1}^{q}\theta_j \varepsilon_{t-j}

Here,

$p$ =Number of AR terms
$q$ =Number of MA terms

Model Identification Steps

| Step | Action |

|------|--------|

| 1 | Plot the series — look for trend, seasonality |

| 2 | Test stationarity — ADF and KPSS tests |

| 3 | Difference if needed — determine d |

| 4 | Examine ACF/PACF — identify p and q |

| 5 | Estimate model parameters |

| 6 | Diagnostics — check residuals |

| 7 | Forecast and evaluate |

AR vs MA

ACF cuts off -> MA model; look at ACF lag for q
PACF cuts off -> AR model; look at PACF lag for p
Both tail off -> ARMA; use AIC/BIC to compare models

Information Criteria

AIC and BIC

AIC = -2\ln(L) + 2k

Here,

$L$ =Maximized likelihood value
$k$ =Number of parameters
$n$ =Sample size

Lower AIC/BIC indicates a better model. BIC penalizes complexity more heavily.

Residual Diagnostics

After fitting, check that residuals are white noise:

Ljung-Box test: No autocorrelation in residuals
Normality test: Residuals approximately normal
Plot: No patterns in residuals vs. fitted values

Ljung-Box Statistic

Q = T(T+2)\sum_{k=1}^{h}\frac{\hat{\rho}_k^2}{T-k}

Here,

$T$ =Sample size
$h$ =Number of lags tested
$\hat{\rho}_k$ =Sample ACF at lag k

Python Implementation


import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

from statsmodels.tsa.arima.model import ARIMA

from statsmodels.tsa.stattools import adfuller

from statsmodels.graphics.tsaplots import plot_acf, plot_pacf



np.random.seed(42)



# Simulate AR(1) process

n = 300

y = np.zeros(n)

for t in range(1, n):

    y[t] = 0.7 * y[t-1] + np.random.randn()



# Stationarity test

adf = adfuller(y)

print(f"ADF p-value: {adf[1]:.4f}")



# Fit ARIMA(1,0,0) = AR(1)

model = ARIMA(y, order=(1, 0, 0))

results = model.fit()

print(results.summary())



# Diagnostics

print(f"\nLjung-Box p-value: {results.test_serial_correlation('ljungbox', lags=[10])[0]['lb_pvalue'].values[0]:.4f}")



# Forecast

forecast = results.forecast(steps=10)

print(f"\n10-step forecast: {forecast[:5].round(3)}")

Worked Example

Example: Fitting ARIMA(1,1,1)

Given a non-stationary time series with 200 observations:

ADF test: p-value = 0.45 -> non-stationary
First difference $\Delta Y_t$ : ADF p-value = 0.001 -> stationary, so d = 1
ACF of differenced series: Significant spike at lag 1, then cuts off -> q = 1
PACF of differenced series: Exponential decay -> p = 1
Fit ARIMA(1,1,1): AIC = 450.2
Compare with ARIMA(0,1,1): AIC = 455.8 -> ARIMA(1,1,1) is preferred
Ljung-Box test on residuals: p-value = 0.35 -> residuals are white noise
Conclusion: ARIMA(1,1,1) provides a good fit

Forecasting

Point Forecast

\hat{Y}_{T+h|T} = E[Y_{T+h} | Y_1, \ldots, Y_T]

Here,

$h$ =Forecast horizon
$T$ =Last observation time

Forecast accuracy is measured by:

| Metric | Formula | Interpretation |

|--------|---------|---------------|

| MAE | $\frac{1}{h}\sum|Y_t - \hat{Y}_t|$ | Average absolute error |

| RMSE | $\sqrt{\frac{1}{h}\sum(Y_t - \hat{Y}_t)^2}$ | Penalizes large errors |

| MAPE | $\frac{100}{h}\sum\left|\frac{Y_t - \hat{Y}_t}{Y_t}\right|$ | Percentage error |

Key Takeaways

Summary: ARIMA Models

ARIMA(p,d,q) combines AR, differencing, and MA components
Use ACF/PACF to identify p and q; use ADF/KPSS to determine d
AR(p): PACF cuts off at lag p
MA(q): ACF cuts off at lag q
Always check residual diagnostics (Ljung-Box, normality)
Use AIC/BIC to compare non-nested models
Over-differencing can introduce unnecessary dependence

ARIMA Models — Complete Guide

ARIMA Models — Complete Guide

Combining Autoregression, Integration, and Moving Average

DfARIMA(p,d,q) Model

ARIMA(p,d,q)

Component Models

AR(p) — Autoregressive

AR(p) Model

MA(q) — Moving Average

MA(q) Model

ARMA(p,q)

ARMA(p,q) Model

Model Identification Steps

Information Criteria

AIC and BIC

Residual Diagnostics

Ljung-Box Statistic

Python Implementation

Worked Example

Example: Fitting ARIMA(1,1,1)

Forecasting

Point Forecast

Key Takeaways

Summary: ARIMA Models

Related Topics

Premium Content

Need Expert Statistics Help?