Prediction Standard Errors

Again like last note we fix ${\displaystyle \theta }$ and attempt to calculate $$V_i(\theta )=\mathrm{Var}(y_{n+i}|y_1,\cdots ,y_n,\theta ),i=1,2,\cdots$$
Then the prediction standard error of ${\displaystyle y_{n+i}}$ can be ${\displaystyle \sqrt{V_i(\hat{\theta })}}$ (${\displaystyle \hat{\theta }}$ comes from conditional MLE). It turns out that it is difficult to directly setup a recursion for ${\displaystyle V_i(\theta )}$, instead we get the recursion by working with conditional covariance matrices of ${\displaystyle y_{n+1},\cdots ,y_{n+k}}$ (given ${\displaystyle \theta }$ and data) for ${\displaystyle k=1,2,\cdots }$ (review 19 Covariance Matrices)

Denote $${\mathrm{\Gamma }}_k(\theta )=\mathrm{Cov}(\left(\begin{array}{c}{y}_{n+1}\\ ⋮\\ y_{n+k}\end{array}\right)|\theta ,y_1,\cdots ,y_n).$$ The ${\displaystyle (i,j)}$ element of ${\displaystyle {\mathrm{\Gamma }}_k(\theta )}$ is ${\displaystyle \mathrm{Cov}(y_{n+i,}{y}_{n+j}|y_1,\cdots ,y_n,\theta )}$. The diagonal elements are ${\displaystyle V_1(\theta ),\cdots ,V_k(\theta )}$. Note that $${\mathrm{\Gamma }}_1(\theta )=\mathrm{Var}(y_{n+1}|y_1,\cdots ,y_n,\theta )={\sigma }^2.$$
We also know $${\mathrm{\Gamma }}_k(\theta )=\left(\begin{array}{cc}{\mathrm{\Gamma }}_{k-1}(\theta )& {\gamma }_{k1}(\theta )\\ {\gamma }_{k1}^{\mathrm{T}}(\theta )& V_k(\theta )\end{array}\right),$$ where $$\begin{array}{rl}{\gamma }_{k1}(\theta )& =\mathrm{Cov}(\left(\begin{array}{c}{y}_{n+1}\\ ⋮\\ y_{n+k-1}\end{array}\right)|\theta ,y_1,\cdots ,y_n)\\ & =\mathrm{Cov}(\left(\begin{array}{c}{y}_{n+1}\\ ⋮\\ y_{n+k-1}\end{array}\right),\sum _{i=1}^{k-1}{a}_i{y}_{n+i}|\theta ,\mathrm{data}),\end{array}$$ where $$a_i=\{\begin{array}{rl}& {\varphi }_{k-i},k-p\le i\le k-1,\\ & 0,1\le i<k-p.\end{array}$$
Thus if ${\displaystyle a}$ is the ${\displaystyle (k-1)\times 1}$ vector with entries ${\displaystyle a_1,\cdots ,a_{k-1}}$, $${\gamma }_{k1}(\theta )=\mathrm{Cov}(\left(\begin{array}{c}{y}_{n+1}\\ ⋮\\ y_{n+k-1}\end{array}\right),a^{\mathrm{T}}\left(\begin{array}{c}{y}_{n+1}\\ ⋮\\ y_{n+k-1}\end{array}\right)|\theta ,\mathrm{data})={\mathrm{\Gamma }}_{k-1}(\theta )a.$$
Further $$V_k(\theta )=\mathrm{Var}(\sum _{i=1}^{k-1}{a}_i{y}_{n+i}|\theta ,\mathrm{data})+{\sigma }^2=a^{\mathrm{T}}{\mathrm{\Gamma }}_{k-1}(\theta )a+{\sigma }^2.$$ And therefore $${\mathrm{\Gamma }}_k(\theta )=\left(\begin{array}{cc}{\mathrm{\Gamma }}_{k-1}(\theta )& {\mathrm{\Gamma }}_{k-1}(\theta )a\\ a^{\mathrm{T}}{\mathrm{\Gamma }}_{k-1}(\theta )& a^{\mathrm{T}}{\mathrm{\Gamma }}_{k-1}(\theta )a+{\sigma }^2\end{array}\right).$$

In practice, we run this recursion with ${\displaystyle {\hat{\theta }}_{\mathrm{MLE}}}$.