The problem posits the simple linear regression model

The simple correlation coefficient is

1. Notice that there is no presumption about causation between x and y. This is quite different from the coefficient of determination in a regression model. In a regression model y is always understood to be the dependent variable. The coefficient of determination will depend on the choice of left hand side variable. However, if y is taken to be the dependent variable, then the square of the correlation coefficient and the coefficient of determination will be equal in a simple linear regression model. If you wanted to use this fact in your answers then you need to prove the equality first.

2. When told to prove or demonstrate something you cannot use the result to be proven in the proof!

3. When told to prove or demonstrate something you cannot pull results out of thin air without showing where the result came from or giving attribution.

The Likelihood Ratio Test for Linear Restrictions on the Regression Coefficients

For the maintained model we have

The maximum likelihood estimator of the error variance under the null hypothesis is

Making the substitution, the log likelihood under the null reduces to

For the restricted model, ω, we have by analogy

The likelihood ratio statistic is

Making the substitutions

The maximum likelihood estimator of the intercept is

Making this substitution and doing some rearranging gives

In the next steps you should expand the square in the denominator, then multiply the numerator and denominator by the inverse of the sum of squared deviations of y about its mean. If you do the algebra correctly then you will obtain

The maximum likelihood estimator for the slope coefficient is

Substitute this expression into the LR

With some rearranging and applying the definition of the correlation coefficient you will get

Which was the desired result.

The Wald Test for Linear Restrictions

We begin by making use of the definition of the Wald test.

The estimate of the variance of the ML estimator for the slope is

Utilizing some of the steps from the answer to the LR question (substituting in for the estimators for the intercept and slope) we can rewrite the estimate of the error variance in the following way

Making the substitution into the Wald statistic

With some cancellations and rearranging you will obtain

Which was the desired result.

The Lagrange Multiplier Test for Linear Restrictions

Recall the likelihood function from the section on the likelihood ratio test. Differentiate that function with respect to each of the unknown parameters.

(a) We will also need the inverse of the negative of the information matrix. Recall that the information matrix is the Hessian for the likelihood function. Since (1) and (3) are both zero when evaluated at the restrictions,

,

we only need one term in the information matrix. Namely, after first substituting the ML estimator for the intercept into (2),

The second equality in (4) comes from evaluating the derivative at the restrictions.

(b) From the observation in (a) and the definition of the LM test in the text we can write

From (2) we can write

Plugging into the test statistic

Which is the desired result.