Asymptotic Results in Basic Linear Model

#econometrics #economics

Oh, Hyunzi. (email: wisdom302@naver.com)
Korea University, Graduate School of Economics.
2024 Spring, instructed by prof. Kim, Dukpa.

Main References

Kim, Dukpa. (2024). "Econometric Analysis" (2024 Spring) ECON 518, Department of Economics, Korea University.
Davidson and MacKinnon. (2021). "Econometric Theory and Methods", Oxford University Press, New York.

Model and Assumptions

The model remains the same as before: where and . We assume are unknown.

In matrix notation, The least-squares estimate of is

Recall the previous assumptions

Assumption 18 (Classic Assumptions).

A1) , .
A2) The model relating and is linear and given by .
A3) and .
A4) , : homoskedasticity in error term.
A5) , : independently distributed error term.

and

Assumption 1 (assumption-Normality).

A-N) the errors are normally distributed. i.e. and

For the rest of the section, we replace the original classic assumptions into much weaker ones, allowing for much more flexible application.

Unbiased	Gauss Markov Thm	Normality	Consistency	Asymptotic Normality
A1)	A1)	A1)	A1*)	A1**)
A2)	A2)	A2)	A2)	A2)
A3)	A3)	A3)	A3)	A3)
	A4&5)	A4&5)	A4&5*)
		A-N)
			A6)
			A7)
			A8)

Here, in consistency, the assumptions are relaxed into:

A1) : strict exogeneity, errors should not correlated with the regressors for any leads and lags.
A1*) : predeterminedness, errors and the regressors are uncorrelated only contemporaneously.

while A1 is stronger version to imply the all A1, A4 and A5 (see ^b315f9 Remark 7 (A1** implies A1, A4, A5, A6 and A8)).

A4&5) : homogeneity and independence between the errors.
A4&5*) : allowing for heterogeneity and correlation between the errors.

Note that must be a non-singular matrix, but can be some unspecified matrix.

For Assumption 6~8, they follow some form of Econometric Analysis/Asymptotics > Theorem 1 (weak law of large numbers), in more general sense.

From A6, denotes the limit of averaged expectations, since which implies that A6 allows the case when changes over time.
From A7, is similar to A6, where performs as a weighted sum of over time.
From A8, which allows the changes in over time.

Note that from the definition they are symmetric and idempotent by the definition.

Consistency

Definition (consistency).

A sequence of estimators is consistent for if when is the true.

Assumption (ASM for consistency).

A1*) : predeterminedness.
A2) .
A3) : full rank.
A4&5*) , where is a non-singular matrix.
A6) , where is a non-singular matrix.
A7) , where is a non-singular matrix.
A8) is the average of variances over .

Theorem (consistency of least-square estimator).

Under A1*, A2, A3, A4&5*, A6, and A7, we have

Proof.From least-square estimator, First, using A6, for a non-singular matrix we have Secondly, from A1*, we have , thus and from A4&A5* () and A7, which means by Convergence of Random Variables > Definition 15 (mean-square convergence) and by Convergence of Random Variables > Lemma 17 (mslim implies plim), Therefore, we have i.e. which completes the proof. □

Theorem (consistency of variance estimator).

Under A1*, A2, A3, A4&5*, A6, A7, and A8, we have

Proof.From the definition of , we have note that and from the proof of ^2f0b7f Theorem 3 (consistency of least-square estimator), we have

Therefore, we have i.e. . □

Remark (consistency and unbiasedness).

There is no implication between unbiasedness and consistency. i.e. unbiasedness does not imply consistency and vice versa.

Asymptotic Normality

Assumption (ASM for asymptotic normality).

A1**) is an sequence with and .
A2) .
A3) : full rank.

Remark (A1** implies A1, A4, A5, A6 and A8).

Note that the assumption A1** implies A1, A4, A5, A6 and A8, where

A1**) is an sequence with and .

Proof.First, we show that A** is a strong enough to imply A1, A4, and A5: where the second equations hold by the of .

Also, remark that A6 and A8 in ^6b0642 Assumption 2 (ASM for consistency) are not needed since they are true by Econometric Analysis/Asymptotics > Theorem 6 (Kolmogorov's Theorem).

First, A6 can be driven as and A7 can be as and lastly, which are hold by Econometric Analysis/Asymptotics > Theorem 1 (weak law of large numbers). □

Lemma (Cramer-Wold device in least-squares estimator).

Let and let . Then under ^9b2df3 Assumption 6 (ASM for asymptotic normality), we have or equivalently, where .

Proof.Note that by A1**, and , we have and

Also, note that and

Thus, we have Now, using Econometric Analysis/Asymptotics > Theorem 12 (Cramer-Wold device), for any real vector such that , we have And since , by Econometric Analysis/Asymptotics > Theorem 8 (Lindeberg-Levy CLT), we have Therefore, we have or This completes the proof. □

Theorem (asymptotic normality of least-squares estimator).

Under A1**, A2, and A3, we have

Proof.From note that where by ^b315f9 Remark 7 (A1** implies A1, A4, A5, A6 and A8), and by ^7e974d Lemma 8 (Cramer-Wold device in least-squares estimator).

Therefore, by Convergence of Random Variables > Theorem 25 (Slutsky theorem), we have where the last equation holds since , since . which completes the proof. □

Remark (approximation).

Since is just a constant, if is large enough, we have then, which is the same results from Inferences in Linear Regression > Theorem 3 (distribution of least square estimates), obtained without assuming that the errors are normally distributed.

Inferences

T-Statistics

Recall the definition of t-statistics form chapter 5:

Theorem 4 (t-test).

Under A1~A5 A1~A5 and A-N A-N, we have where

Where

Proposition (asymptotic t-stat).

Under ^9b2df3 Assumption 6 (ASM for asymptotic normality), the -statistics is

Proof.Note that from Inferences in Linear Regression > Theorem 4 (t-test). Then, by multiplying on both numerator and denominator of the right hand side, And the numerator follows by ^0accbb Theorem 9 (asymptotic normality of least-squares estimator), where the denominator converges to by ^bffde9 Theorem 4 (consistency of variance estimator) and ^b315f9 Remark 7 (A1** implies A1, A4, A5, A6 and A8), implying that by Convergence of Random Variables > Theorem 23 (Continuous Mapping Theorem) and Convergence of Random Variables > Remark 26 (alternative form of Slutsky).

Finally, by Convergence of Random Variables > Theorem 25 (Slutsky theorem), we have which completes the proof. □

Accordingly, the confidence interval is given on a standard normal distribution.

Proposition (asymptotic confidence interval).

An approximate confidence interval for is given by where is the percentage point of a distribution and is the standard error of .

Note that

Actual size: computed from the finite sample distribution.
Nominal size: computed from the limiting distribution.

Since we don't know the actual distribution of the finite sample when the true error does not follow the normal distribution, the actual size is not computable in many cases. Here, nominal size is computed from the standard normal in ^dbfb39 Proposition 12 (asymptotic confidence interval), and if the sample size is big enough, the nominal size approximately same to the actual size.

liberal test: if actual size is greater than nominal size.
conservative test: if actual size is smaller than nominal size.

Additionally, the difference between the actual and nominal size is called 'size distortion' in the test. This difference is usually calculated from Monte-Carlo simulation.

Remark (consistent test in asymptotic).

From Inferences in Linear Regression > Remark 5 (decomposition of t-stat), we have where and Thus, if is true, then i.e. the test is consistent.

F-Statistics

Recall the F-statistics from chapter 5:

Theorem 8 (f-test).

Under A1~A5 A1~A5 and A-N A-N, we have where

where and is matrix and is vector given by the researcher. The t-test check whether the estimated is significantly different from .

Proposition (asymptotic f-statistics).

Under ^9b2df3 Assumption 6 (ASM for asymptotic normality), the -statistics is

Proof.Note that since we have where under the , as , we have Also, remark that by ^bffde9 Theorem 4 (consistency of variance estimator) and ^b315f9 Remark 7 (A1** implies A1, A4, A5, A6 and A8).

Thus we have where the last equation is by Normal Distribution Theory > ^ca7ceb Normal Distribution Theory > Lemma 9 (multivariate normal and chi-squared distribution).

Therefore, we have which completes the proof. □

Remark (wald statistic).

From the f-statistic the Wald statistic is defined as