Wold Decomposition

Oh, Hyunzi. (email: wisdom302@naver.com)
Korea University, Graduate School of Economics.

Main References

Kim, Dukpa. (2022). "Time Series Econometrics" (2022 Fall) ECON 512, Department of Economics, Korea University.
Hamilton, J. D. (1994). "Time Series Analysis". Princeton University Press.

Linear Process

Definition (linear process).

The process is said to be a general linear process if it can be expressed in the form where is a purely random stochastic process.

Alternatively, we can understand as a thus, it is unclear whether the sum of infinitely many random variables is finite. Therefore, we need to find a set of conditions on and , to guarantee that is a well behaved random variables.

Convergence of Linear Process

From Hilbert and Lp spaces > ^3f9379 Hilbert and Lp spaces > Definition 9 (Lp Space; Lebesque Space), space is defined as i.e. a collection of random variables with finite variance, where its inner product inner product is and the associated norm of is Using the inner product of , we can define the convergence of a random variable as follows:

Definition (convergence of random variable).

A sequence of random variables is convergent to the limit , i.e. if

and under , we have

Definition (cauchy).

A sequence of random variables is said to be cauchy if as .

Then, we have the elementary theorem we've learned in the analysis course.

Proposition (converging sequence is caucy).

If , then is also a cauchy sequence. However, the converse does not always hold.

Proof.Let be given. Since is assumed to be converging, there exists some such that . Thus there exists some positive integer such that thus by triangle inequality, for all , we have which completes the proof. □

Definition (complete space).

The space is called as complete space if every cauchy sequence converges to some member in the space.

Proposition (L2 is complete).

space is complete space.

The proof of the proposition is omitted (you can refer to some real analysis textbook if you are interested). Note that since is a complete inner product space, thus it is a Hilbert space.

Stationary Linear Process

Theorem (stationary linear process).

If and if , then the series converges in mean square and is stationary.

Proof.First, define and it is finite if . Thus the variance of is finite, meaning that . By ^6ee1f3 Proposition 6 (L2 is complete), there exists some such that Assume the case when . Then, since where the last convergence holds since Therefore is cauchy, and thus it converges to some member in . This implies that which is the first part of the theorem.

Next, we prove the stationarity of . Remark Hilbert and Lp spaces > ^415796 Hilbert and Lp spaces > Theorem 4 (continuity of inner product), i.e. Without loss of generality, assume . Using the continuity of inner product, we have and thus which is the mean stationarity.

Note that the square-summability of the coefficient of linear process is crucial to prove the stationarity.

Theorem (a.s. convergence of linear process).

If is any sequence of random variables such that and , then the linear process converges absolutely with probability one. In addition, if , then the series converges in mean square as well.

Proof.Before begin the proof, remark Monotone Convergence Theorem > ^dba38f Monotone Convergence Theorem > Theorem 2 (monotone convergence theorem), i.e. for an increasing sequence of non-negative measurable functions , if , then .

Since we have , i.e. uniformly bounded first moments of , using MCT, we have thus with probability one, both and are finite.

Now, we prove when is additionally assumed. Similar to the proof of ^b108d9 Theorem 7 (stationary linear process), by letting and , we have as . □

Note that compared to ^b108d9 Theorem 7 (stationary linear process), ^cea55f Theorem 8 (a.s. convergence of linear process) requires much weaker conditions.

Remark (aslim and mslim).

The almost sure and mean square limits in ^cea55f Theorem 8 (a.s. convergence of linear process) must be the same. In general, since and by Fatou's lemma of

Autocovariance Generating Function

Definition (autocovariance generating function).

Let is a stationary process with autocovariance function . Then the autocovariace generating function (AGF) of is defined by

Definition (Lag polynomial).

We denote as a lag operator which is and we simply denote as the lag polynomial where

Remark (AGF of white noise process).

Note that AGF of white noise process is a constant, since its is only when . Furthermore, if and there exists such that then we have

Wold Decomposition

Theorem (wold's decomposition).

Any stationary process can be expressed in the form where

and are uncorrelated process
is regular with one-sided linear representation with , and is a white noise process uncorrelated with given by i.e. is the forecast error from the optimal predictor of using the past information.
can be predicted from its own past with zero variance.

Wold's decomposition is a very strong result since it says that any stationary process can be well approximated by a linear process. Note that cannot be a constant. For example, the and can be random variable. The important thing is that is drawn at the beginning of the process and fixed after on.

Estimating Autocovariance Function

Definition 6 (autocovariance function).

If is a process such that for each , then the autocovariance function (ACF) of is defined by

The two most popular estimators are note that the both estimators are biased, the bias is bigger for the latter. However, the bias is asympotically negligible for both of them.

Here are some characteristics of the two estimatots:

Remark (preferred estimator of ACF).

Note that in general, is preffered, since is positive-semidefinite, a property shared by . Also, it has smaller MSE,

Remark (unbiased estimator of ACF).

Note that the following estimator is unbiased

Remark (estimator for ACRF).

Theorem (asympotic normality of ACRF estimator).

If is a stationary process where and , then for each , we have where , , and is the covariance matrix where element is given by Bartlett's formula

Remark (covarince matrix of WN).

If is WN, then we have

Note that ^e1cab5 Remark 18 (covarince matrix of WN) is the gound for the test to see if the data has serial correlation. Formally, let

Definition (box-pierce statistics).

The Box-Pierce statistic is defined as where the asymptotic hold under .

The asymptotic can be intuitively understood since follows WN, we have and . Then, . Thus by Normal Distribution Theory > ^46ff8a Normal Distribution Theory > Proposition 11 (normal and chi-squared distribution),

Definition (lijung-box statistic).