Statistics 102H: MLE's (Statistical Inference by S. D. Lilvey

Administrivia

• data analysis due today
• I'll quickly read over your analysis to see if there are any MAJOR statistical prolbem. If you have any minor statistical issues, please not them on the cover and I'll email you a response.
• We will not cover sections 4.3 and 4.7 of handout

Review: taking the deravitives

• first derivative is called the score
• it can be expanded as a sum of derivatives of each term
• second dervative is called the Fisher Information
• It typically acts like a constant (i.e. it doesn't depend very much on the data itself.)

First useful example

• Suppose there are outliers in the data
• You can handle it using ideas of the first part of hte course
• Now lets do it carefully.
• Need a distribution that generates outliers: Cauchy
• Compute the score (i.e. use mathematica)

         In[31]:= f[t_]
  
                        1
         Out[31]= -------------
                              2
                  1 + (x - t_)
  
         In[32]:= l[t_]
  
                            1
         Out[32]= Log[-------------]
                                  2
                      1 + (x - t_)
  
         In[33]:= l'[t_]
  
                   2 (x - t_)
         Out[33]= -------------
                              2
                  1 + (x - t_)
  
         In[34]:= l''[t_]
  
                                              2
                       -2           4 (x - t_)
         Out[34]= ------------- + ----------------
                              2                2 2
                  1 + (x - t_)    (1 + (x - t_) )
  

• Notice: large values are down-weighted! Just like you do when you remove them!

Major theorems

• consistency (section 4.4 and 4.5): theta-hat --> theta (in prob)
• efficiency (section 4.6): hits Cramer-Rao lower bound
• normality (section 4.6): P(theta-hat < z) --> normal distribution

• consisitency: MLE will be with in spitting distance of the truth. IN other words, it won't be fooled into an alternative global minimum.
• efficiency: No other estimatory that is different than the MLE has a lower standard error
• normaility: once you compute the MLE, you can use the Z-statistics to give CI's and hypothesis testing.

Homework

• Page 84 of handout:
  • 4.1 just compute the MLEs for each case
  • 4.2
  • 4.3 (skip the "not idenifiable" part)
  • 4.6 (skip simple approximation)
  • BONUS: 4.7 is hard! It requires doing some geometry.