Statistics 102H: FPP (or FPPA) Freedman's book on Statistics

Administrivia

Turn in first part of your project (if you didn't email it).
be sure to include an email address on it!
I'm covering philosophy of testing. Read book for details! (You ARE responsible for details.)

Chapter 26: Tests of significance

Review:

Box model for where our data comes from.
Average of sample is property of interest
Sample average (called X-bar) is close to population average (called mu)
How close? SE = SD/Sqrt(n) close.
In particular: (X-bar - mu)/SE has approximately a normal distribution
Here is where typical classes start looking stuff up in tables. Instead, I'll have you memorize the table:
- P(Z <-3) = 1/1000
- P(Z <-2) = 1/40
- P(Z <-1) = 1/6
- P(Z < 0) = .5
- P(Z < 1) = 5/6
- P(Z < 2) = 39/40
- P(Z < 3) = .999
Lucky you! (2/3, 19/20, 99.8 are the way I remimber them)

Story time

Consider the following story of two opinions about an upcoming tax cut:

New taxes: "revinue neutral" or not?
Draw 100 returns from previous years. Recompute their tax payment.
X-bar = -$219. SD = $725.
First conversation: SD vs SE.
Second conversation: test statistic: (mean - target)/SE = -3. corresponds to a 1/1000 chance of seeing a mean that far away from 0. Yikes!

Finally, we will be able to define:

null hypothesis
alternative hypothesis
test statistic
p-value

Null and alternative

Here the null was: the average in the box = $0. The alternative the tresury worried about was that the average was less than zero. (Tax payers might worry about another alternative.)

Null claims that the observed difference are due to chance alone. The alternative says the differences are real.

Test statistic

A test statistic measures the difference between the data and what is expected under then Null.

Typically: test statistic is normalized to look approximately normal.

P-value = observed significance level.

P-value is chance of getting a test statistic this strange under then Null. NOT, the chance of the null being correct! (This difference should now be obvious to you. If it isn't read the chapter carefully!)

Proofs and language

You can disprove the null

Suppose p-value = 1/Million
Suppose you do another test and it is again 1/Million
Overall p-value = 1/Trillion
Do you think the null is still true?

So we agree, you CAN disprove a null.

Can you prove a null true?

you do a test and the p-value = .6
you do a test and the p-value = .2
you do a test and the p-value = .8
you do a test and the p-value = .5
you do a test and the p-value = .9
You combine all the data and the p-value = .3
Is the null true? Suppose the total amount of data was small?

All you can say is "I've failed to disprove the null." This is very different than saying you have proven the null. It could be a personal flaw, or it could be the null is true. You will never know.

Example of putting it all together

Think, share, pair: Problem 11 on page 489.

Homework

Read chapter 26. In particular, read sections 5 - 8. I didn't cover them, but you are responsable for them.
What will your test statistic be in your project? (You might not be able to answer this yet if your project involves a regression)

Last modified: Mon Apr 7 08:18:19 2003