Do students give higher evaluations in the morning?
Suppose we are interested in whether students give higher class
evaluations in the morning than in the afternoon. (Theory is based on
anectodotal studies.) How can we test this claim?
Exp 1: Abba teaches morning classes for next 10 years. Bob
teaches afternoon classes for next 10 years. n=30, m=30.
Exp 2: Abba does 5 years in morning (while Bob does 5 years in
afternoon) then then switches with Bob. n = 30, m =30.
Alternative theory: n = 2, m = 2. (Maybe Bob is completely
tiard after his noon bike ride and hence doesn't teach well in
the afternoon.)
Exp 3: Abba teaches both morning and afternoon classes each
semester. So does Bob, Charles, Dean, Edward, Fred, George.
Look at bob_afternoon - bob_morning and abba_afternoon -
abba_morning. One sample: n = 7.
Alternative theory: morning students grade easier than
afternoon students.
Aside: do we care?
Under this theory we would still choose to teach in the
morning.
BUT, the deparmtnet can't do it on mass since that will
drive the afternoon students to the morning.
Exp 4: Randomly assign students to classes. Solves all
problems but practical ones! (Ethics, scheduling conflects,
adminstration not liking it.) n = 1000s. m = 1000s.
So what are the control variables of the deparment? When
classes are taught.
Exp 5: Run only morning classes for 5 years, then run only
afternoon classes for 5 years. See difference.
Alternative theory: Students are getting grumpier/easier with
time.
Exp 6: Randomly decide each semester to run only morning
classes or only afternoon classes. See difference. For 10
years we would get an n=10, m=10.
Handout on DOE
Key items in design
H0, H1 as science
Xs, Ys
what is a "row"?
randomization, controls, data collection
models
H0, H1 as statistics
n
bad things: missing data, costs, time, interium analysis
for stopping
Some examples
plan, model, key statistical question
Rogaine vs. Gro-againe
Science: prevention of balding
what to measure?
units of analysis. Parts of heads, or whole heads?
randomization/data collection
statistical hypothesis (beats rogaine, equal to or
different than?)
Missing data. Bored with poor results, moving out of
area, etc.
Sample size calculations
signficance, power, sample size, signal/noise are key
properties
Goal: determine sample size from other items
One sample problem
Alpha says how far away x-bar has to be to be
significant
Beta for some delta says how far away x-bar has to be
from interesting point