-
Notifications
You must be signed in to change notification settings - Fork 167
/
9-arima_c.qmd
201 lines (165 loc) · 8.12 KB
/
9-arima_c.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
---
title: "ETC3550/ETC5550 Applied forecasting"
author: "Ch9. ARIMA models"
institute: "OTexts.org/fpp3/"
pdf-engine: pdflatex
fig-width: 7.5
fig-height: 3.5
format:
beamer:
theme: monash
aspectratio: 169
fontsize: 14pt
section-titles: false
knitr:
opts_chunk:
dev: "cairo_pdf"
include-in-header: header.tex
execute:
echo: false
message: false
warning: false
---
```{r setup, include=FALSE}
source("setup.R")
library(patchwork)
library(purrr)
```
## Point forecasts
1. Rearrange ARIMA equation so $y_t$ is on LHS.
2. Rewrite equation by replacing $t$ by $T+h$.
3. On RHS, replace future observations by their forecasts, future errors by zero, and past errors by corresponding residuals.
Start with $h=1$. Repeat for $h=2,3,\dots$.
## Prediction intervals
\vspace*{0.2cm}\fontsize{14}{15}\sf
\begin{block}{95\% prediction interval}
$$\hat{y}_{T+h|T} \pm 1.96\sqrt{v_{T+h|T}}$$
where $v_{T+h|T}$ is estimated forecast variance.
\end{block}\pause\vspace*{-0.3cm}
* $v_{T+1|T}=\hat{\sigma}^2$ for all ARIMA models regardless of parameters and orders.\pause
* Multi-step prediction intervals for ARIMA(0,0,$q$):
\centerline{$\displaystyle y_t = \varepsilon_t + \sum_{i=1}^q \theta_i \varepsilon_{t-i}.$}
\centerline{$\displaystyle
v_{T|T+h} = \hat{\sigma}^2 \left[ 1 + \sum_{i=1}^{h-1} \theta_i^2\right], \qquad\text{for~} h=2,3,\dots.$}
## Prediction intervals
* Prediction intervals **increase in size with forecast horizon**.
* Prediction intervals can be difficult to calculate by hand
* Calculations assume residuals are **uncorrelated** and **normally distributed**.
* Prediction intervals tend to be too narrow.
* the uncertainty in the parameter estimates has not been accounted for.
* the ARIMA model assumes historical patterns will not change during the forecast period.
* the ARIMA model assumes uncorrelated future \rlap{errors}
## Seasonal ARIMA models
| ARIMA | $~\underbrace{(p, d, q)}$ | $\underbrace{(P, D, Q)_{m}}$ |
| ----: | :-----------------------: | :--------------------------: |
| | ${\uparrow}$ | ${\uparrow}$ |
| | Non-seasonal part | Seasonal part of |
| | of the model | of the model |
where $m =$ number of observations per year.
## Seasonal ARIMA models
E.g., ARIMA$(1, 1, 1)(1, 1, 1)_{4}$ model (without constant)\pause
$$(1 - \phi_{1}B)(1 - \Phi_{1}B^{4}) (1 - B) (1 - B^{4})y_{t} ~= ~
(1 + \theta_{1}B) (1 + \Theta_{1}B^{4})\varepsilon_{t}.
$$\pause\vspace*{-1cm}
\setlength{\unitlength}{1mm}
\begin{footnotesize}
\begin{picture}(100,25)(-5,0)
\thinlines
{\put(15,22){\vector(0,1){6}}}
{\put(32,10){\vector(0,1){18}}}
{\put(48,22){\vector(0,1){6}}}
{\put(66,10){\vector(0,1){18}}}
{\put(97,22){\vector(0,1){6}}}
{\put(115,10){\vector(0,1){18}}}
{\put(0,17){$\left(\begin{array}{@{}c@{}} \text{Non-seasonal} \\ \text{AR(1)}
\end{array}\right)$}}
{\put(22,5){$\left(\begin{array}{@{}c@{}} \text{Seasonal} \\ \text{AR(1)}
\end{array}\right)$}}
{\put(35,17){$\left(\begin{array}{@{}c@{}} \text{Non-seasonal} \\ \text{difference}
\end{array}\right)$}}
{\put(55,5){$\left(\begin{array}{@{}c@{}} \text{Seasonal} \\ \text{difference}
\end{array}\right)$}}
{\put(85,17){$\left(\begin{array}{@{}c@{}} \text{Non-seasonal} \\ \text{MA(1)}
\end{array}\right)$}}
{\put(105,5){$\left(\begin{array}{@{}c@{}} \text{Seasonal} \\ \text{MA(1)}
\end{array}\right)$}}
\end{picture}
\end{footnotesize}
\vspace*{10cm}
## Seasonal ARIMA models
E.g., ARIMA$(1, 1, 1)(1, 1, 1)_{4}$ model (without constant)
$$(1 - \phi_{1}B)(1 - \Phi_{1}B^{4}) (1 - B) (1 - B^{4})y_{t} ~= ~
(1 + \theta_{1}B) (1 + \Theta_{1}B^{4})\varepsilon_{t}.
$$\vspace*{-1cm}
All the factors can be multiplied out and the general model
written as follows:\vspace*{-0.7cm}
\begin{align*}
y_{t} &= (1 + \phi_{1})y_{t - 1} - \phi_1y_{t-2} + (1 + \Phi_{1})y_{t - 4}\\
&\text{}
- (1 + \phi_{1} + \Phi_{1} + \phi_{1}\Phi_{1})y_{t - 5}
+ (\phi_{1} + \phi_{1} \Phi_{1}) y_{t - 6} \\
& \text{} - \Phi_{1} y_{t - 8} + (\Phi_{1} + \phi_{1} \Phi_{1}) y_{t - 9}
- \phi_{1} \Phi_{1} y_{t - 10}\\
&\text{}
+ \varepsilon_{t} + \theta_{1}\varepsilon_{t - 1} + \Theta_{1}\varepsilon_{t - 4} + \theta_{1}\Theta_{1}\varepsilon_{t - 5}.
\end{align*}
\vspace*{10cm}
## Seasonal ARIMA models
The seasonal part of an AR or MA model will be seen in the seasonal lags of
the PACF and ACF.
\alert{ARIMA(0,0,0)(0,0,1)$_{12}$ will show:}\vspace*{-0.2cm}
* a spike at lag 12 in the ACF but no other significant spikes.
* The PACF will show exponential decay in the seasonal lags; that is, at lags 12, 24, 36, \dots.
\alert{ARIMA(0,0,0)(1,0,0)$_{12}$ will show:}\vspace*{-0.2cm}
* exponential decay in the seasonal lags of the ACF
* a single significant spike at lag 12 in the PACF.
## ARIMA vs ETS
\fontsize{14}{16}\sf
* Myth that ARIMA models are more general than exponential smoothing.
* Linear exponential smoothing models all special cases of ARIMA models.
* Non-linear exponential smoothing models have no equivalent ARIMA counterparts.
* Many ARIMA models have no exponential smoothing counterparts.
* ETS models all non-stationary. Models with seasonality or non-damped trend (or both) have two unit roots; all other models have one unit \rlap{root.}
\vspace*{10cm}
## ARIMA vs ETS
```{r venn, echo=FALSE}
#| fig-height: 3
#| fig-width: 5
library(latex2exp)
cols <- c(ets = "#D55E00", arima = "#0072b2")
tibble(
x = c(-1, 1),
y = c(-0.5, -0.5),
labels = c("ets", "arima"),
) |>
ggplot(aes(colour = labels, fill = labels)) +
ggforce::geom_circle(aes(x0 = x, y0 = y, r = 1.6), alpha = 0.3, linewidth = 1) +
scale_colour_manual(values = cols) +
scale_fill_manual(values = cols) +
coord_fixed() +
guides(fill = "none") +
annotate("text", label = "ETS models", x = -1.5, y = 1.35, col = cols["ets"], fontface = "bold", size = 6, family = "Fira Sans") +
annotate("text", label = "Combination\n of components", x = -1.3, y = 0.5, col = cols["ets"], size = 2.8, fontface = "bold", family = "Fira Sans") +
annotate("text", label = "9 ETS models with\n multiplicative errors", x = -1.6, y = -0.5, col = cols["ets"], size = 2.8, family = "Fira Sans") +
annotate("text", label = "3 ETS models with\n additive errors and\n multiplicative\n seasonality", x = -1.3, y = -1.4, col = cols["ets"], size = 2.8, family = "Fira Sans") +
annotate("text", label = "ARIMA models", x = 1.5, y = 1.35, col = cols["arima"], fontface = "bold", size = 6, family = "Fira Sans") +
annotate("text", label = "Modelling\n autocorrelations", x = 1.3, y = 0.5, col = cols["arima"], size = 2.8, fontface = "bold", family = "Fira Sans") +
annotate("text", label = TeX("Potentially $\\infty$ models"), x = 1.6, y = -0.6, col = cols["arima"], size = 2.8, family = "Fira Sans") +
annotate("text", label = "All stationary models\n Many large models", x = 1.25, y = -1.5, col = cols["arima"], size = 2.8, family = "Fira Sans") +
annotate("text", label = "6 fully additive\n ETS models", x = 0, y = -0.6, col = "#6b6859", size = 2.8, family = "Fira Sans") +
guides(colour = "none", fill = "none") +
theme_void()
```
## Equivalences
\fontsize{13}{15}\sf
|**ETS model** | **ARIMA model** | **Parameters** |
| :------------ | :-------------------------- | :----------------------------------- |
| ETS(A,N,N) | ARIMA(0,1,1) | $\theta_1 = \alpha-1$ |
| ETS(A,A,N) | ARIMA(0,2,2) | $\theta_1 = \alpha+\beta-2$ |
| | | $\theta_2 = 1-\alpha$ |
| ETS(A,A\damped,N) | ARIMA(1,1,2) | $\phi_1=\phi$ |
| | | $\theta_1 = \alpha+\phi\beta-1-\phi$ |
| | | $\theta_2 = (1-\alpha)\phi$ |
| ETS(A,N,A) | ARIMA(0,0,$m$)(0,1,0)$_m$ | |
| ETS(A,A,A) | ARIMA(0,1,$m+1$)(0,1,0)$_m$ | |
| ETS(A,A\damped,A) | ARIMA(1,0,$m+1$)(0,1,0)$_m$ | |