Classical Tests in MLE

Suppose that the null hypothesis of interest is expressed as where is a vector function, and denotes the number of restrictions we are imposing. Note that we can alternatively express as where is a vector function with a vector argument , which is the number of the free parameters left unrestricted under restrictions.

For example, consider where and we are imposing the restriction of , which is then we have

For the rest of the section, we will follow the notations and the restricted MLE as

Here, we use Lagrangian to derive , by F.O.C. and by denoting : matrix, we have

Alternatively, we can derive using . Define and then solve which gives Later on, we will use the first derivative of as denoted as which is a matrix.

Proof.Given Maximum Likelihood Estimation > ^33437dMaximum Likelihood Estimation > Theorem 8 (asymptotic normality of MLE), we can show that is asymptotically normal around using Econometric Analysis/Asymptotics > ^3c500eEconometric Analysis/Asymptotics > Proposition 14 (delta method).

First, let be some vector between and . Then by taylor's theorem(or mean value theorem), we have and given Maximum Likelihood Estimation > ^46e829Maximum Likelihood Estimation > Theorem 7 (consistency of MLE), we have under the null hypothesis of , since and lies between the two of them.

Then, we have Since under , we have Therefore, by Normal Distribution Theory > ^ca7cebNormal Distribution Theory > Lemma 9 (multivariate normal and chi-squared distribution), we have which completes the proof. □

Note that under , i.e. , the lagrange multiplier should be close to zero given Maximum Likelihood Estimation > ^46e829Maximum Likelihood Estimation > Theorem 7 (consistency of MLE), or the score function should be close to zero at . In other words, if the restriction is non-binding, then the value of the becomes zero.

Proof.Using taylor expansion to the lagrangian equation, there exists some between and , that satisfies Then the first equation can be written as and the second equation as where under the null hypothesis.

In matrix form, we have and thus Then using the inverse of partitioned matrix, we have where the comes from Maximum Likelihood Estimation > ^33437dMaximum Likelihood Estimation > Theorem 8 (asymptotic normality of MLE). Note that stands for the terms that are asymptotically negligible (see Econometric Analysis/Asymptotics > ^b56840Econometric Analysis/Asymptotics > Definition 18 (Op and op)).

Therefore, we have where the last line holds by Normal Distribution Theory > ^ca7cebNormal Distribution Theory > Lemma 9 (multivariate normal and chi-squared distribution). □

Likelihood ratio test check whether the under imposed restriction, the likelihood significantly decrease, meaning it is the wrong hypothesis.

Proof.By the mean value theorem, there exists some between and such that where by the definition of MLE. Thus we have From the proof of Maximum Likelihood Estimation > ^33437dMaximum Likelihood Estimation > Theorem 8 (asymptotic normality of MLE), we have obtained Plugging this back in to our proof, we have Now similarly, obtain the taylor theorem where we have . Then, under , we have Remark that and Thus we have Then, from the previous result, Finally, we have where and Since we have by Maximum Likelihood Estimation > ^33437dMaximum Likelihood Estimation > Theorem 8 (asymptotic normality of MLE), and such that , by Normal Distribution Theory > ^ca7cebNormal Distribution Theory > Lemma 9 (multivariate normal and chi-squared distribution), we have which completes the proof. □