Statistics 541: Sandwich Estimator

Admistrivia

Car data example

Sandwich Estimator

The problem

• heteroskedasticity = fan shaped residuals
• usual estimator is "consistent." (Like who cares?)
• SEs are wrong! (Now this is important.)
• Hypothesis-tests are wrong, CIs are wrong
• Even Bonferonni doesn't work!

Example

• Suppose X mostly equals zero and sometimes equals 1
• Suppose Y = iid N(0,1)
• slope estimate = Y at (X =1)
• Suppose it is heteroskadastic, small variance at zero large at one
• high probably of looking significant, having incorrect SEs, making bad predictions.

First solution: weighted least squares

• suppose Y_i = X_i beta + sigma_i Z_i instead of Y_i = X_i beta + sigma Z_i
• Then Y_i/sigma_i = X_i/sigma_i beta + Z_i
• But this is homoskadastic and we are done
• Where do the sigma_i's come from?
  • theory hopefully
  • estimation possibly. E.g. fit model to Y_i², or (Y_i - Y-hat)². Then use predictions from this model to weight regression.

Second solution: Sandwich estimator.

• Use usual LS estimators for Y
• beta-hat = (X'X)^-1X'Y
• So var(beta-hat) = (X'X)^-1X' var(Y_i) X(X'X)^-1
• Called sandwich estimator since the variance of Y is sandwiched between the two inverses.
• Consistent for true variance of beta-hat

example revisited

• Suppose you fit then compute variance
• Oops! zero variance estimate at (X = 1)
• Better is compute variance, THEN fit

Third solution: Use both!

• First change by doing weighted least squares
• Then use sandwich on resulting Y's
• If your weights are wrong, you should still get good results.

Long, J. S. and Ervin, L. H. (2000), "Using heteroscedastic consistent standard errors in the linear regression model," American statistician, 54, 795 - 806.

White (1980) "A heteroscedastic-consistent covariance matrix estimator and a direct test of heteroskedasticity," Econometrica, 48, 817 - 838.