Dean Foster's Research

Variable selection

Everyone does stepwise regression--statisticians don't approve, but then statisticians don't approve of many things. Edward and I wanted to justify stepwise regression and hopefully find a better way of doing it. Our justification was to consider a risk-ratio. We then optimized this criterion and came up with a proceedure that suggests adding variables if the F-statistic is bigger than 2 log p (basically this is Bonferroni). While we were waiting around for referee's reports we talked to Donoho and Johnstone who also had a paper under review that had the same idea.

"The Risk Inflation Criterion for Multiple Regression"(JSTOR) with E. George, The Annals of Statistics, 22, (1994), 1947 - 1975.

The next big gain in variable selection was to switch from 2 log p to 2 log (p/q). This value is the smallest cut-off that doesn't obviuosly over fit. Decreasing it even a little bit will led to massive overfitting. So the question is, can we prove it doesn't over fit?

Edward and I have approached it from an empirical Bayes perspective. Bob and I have approached it from information theory. Neither approach has lead to a proof that the risk has desirable properties.

"An Information Theoretic Comparison of Model Selection," with Bob Stine .
"Variable selection in data mining: Building a predictive model for bankruptcy," with Bob Stine , (talk).
"Empirical Bayes Variable Selection, (pdf)" with Edward George, Biometrika, 2000, 731 - 747.
"Local Asympotic Coding for Model Selection (pdf)," with Bob Stine , , IEEE Transaction on Information Theory, (1999) 1289 - 1293.
"Honest Confidence Intervals for the Error Variance in Stepwise Regression" Bob Stine .
"Universal Codes for Finite Sequences of Integers Drawn from a Monotone Distribution", with R. Stine and A.J. Wyner, To Appear in the IEEE Transactions on Information Theory, 2002.

Bob Stine and I finally got a risk result for estimators of this type.

"Adaptive variable selection with Bayesian oracles. " with Bob Stine.
Sprinkling in a bit of game theory, led to considering a market for variables. The idea is a player buys the oppertunity to try a variable. If that variable turns out to be significant, then the player is rewarded. The price can be paid in "alpha" or in "bits." These ideas have lead to the following papers:
- Multiple hypothesis testing using the excess discovery count and alpha-investing rules (.pdf) with Robert Stine.
- We are also working on two more papers.

Learning models (Calibration/No-regret/game theory)

Suppose you observe a sequence of events--but you are unwilling to consider a probabilistic model for these events. Can you still come up with good forecasts? Blackwell (1956) and Hannan (1957) showed that this was possible (as did I, but they were first). Rick's and my review paper (1999) discuss all the various people who have independently rediscoved these sorts of results.

In 1991 Rick and I also came up with a scheme that has a property called calibration. The idea of calibration is to make sure that your forecasts make empirical sense. So when you say there is a .2 chance of rain, it should rain empirically 1/5 of the time. This idea (along with our idea of no-regret) has a nice game theoretic applications: calibrated rules converge to correlated equilibria.

"Prediction in the Worst Case,"(JSTOR) The Annals of Statistics, 19, (1991), 1084 - 1090.
"A Randomization Rule for Selecting Forecasts," with Rakesh Vohra, Operations Research, 41, (1993), 704 - 709, with discussion by R. Clemen.
"Asymptotic Calibration," with Rakesh Vohra, Biometrika.
"Calibrated Learning and Correlated Equilibrium," with Rakesh Vohra to appear in Games and Economic Behaviour.
"Introduction to the Special Issue," (in honor of David Blackwell) with R. Vohra, D. Levine, Games and Economic Behavior, (1999), 1 - 7.
"Regret in the On-line Decision Problem," with Rakesh Vohra, Games and Economic Behavior, 1999, 7 - 36.
"A proof of Calibration via Blackwell's Approchability Theorem.", Games and Economic Behavior, 1999, 73 - 79.

My work with Rick showed that one can learn correlated equilibria. But, can one learn the more traditional concept of a Nash equilibrium? Peyton and I extended Jordon's proof that Bayesians can't learn a Nash equilibrium to include Bayesians that do discounting.

"Learning, Hypothesis Testing, and Nash Equilibrium (ps)" (.pdf), with H. P. Young.
"Learning with Hazy Beliefs," (1997) with H. P. Young. (No longer current. See "Hypothesis testing" above.)
"On the Impossibility of Predicting the Behavior of Rational Agents (ps)," (.pdf) with H.P. Young, PNAS, 2001.
"On the Nonconvergence of Fictitious Play in Coordination Games," with H. P. Young, Games and Economic Behaviour, 1998, 79 - 96.

It turns out that you can make a version of calibration that can be used as a coordinating device to discover a NE. It is still as exponentially slow as the exhaustive search done in the "Hypothesis Testing" approach.

Deterministic Calibration and Nash Equilibrium (.pdf) with Sham M. Kakade, COLT, 2004.
"Determinisic calibration with simplier checking rules," with Sham M. Kakade, talk.ps (.pdf), handout.ps (.pdf).

ARCH models

If you look the stock market, there are periods when not much is happening and the prices are stable and there are periods when the prices flucuate all over the place. Dan Nelson had me assist him in his analyis of these ARCH models.

"Continuous Record Asymptotics for Rolling Sample Variance Estimators," with D. Nelson, Econometrica , 64, (1996), 139 - 174.
"Filtering and Forecasting with Misspecified ARCH Models: Making the Right Forecast with the Wrong Model," with D. Nelson, Journal of Econometrics, 67, (1995), 303 - 335.
"Asymptotic Filtering Theory for Univariate ARCH Models,"(JSTOR) with D. Nelson, Econometrica 62, (1994), 1 - 41.

Affirmative action

Rick and I saw a talk by Judge Posner, that was so rabidly free-market that we wanted to write a piece showing that the free-market can't solve all problems. (Posner was argueing that rape is primarlly a search cost issue.) So we wrote this little model which showed that if all agents are rational, a free market is unable to remove arbitary and capricious discrimination. While our paper was under review, we were given a copy of a paper by Coate and Loury which had basically the identical model. Interestingly enough, it had the opposite conclusion.

"An Economic Argument for Affirmative Action," with Rakesh Vohra, Rationality and Society, 4, (1992), 176 - 188, with discussion by G. Loury, by D. Friedman, and by J. Heckman and T. Philipson.

Evolution and games

In 1988, Peyton Young and I came up with the idea of merging population genetics with game theory. This had be done for deterministic dynamics already--we just added noise to the existing definition. This is my most frequently cited paper.

"Stochastic Evolutionary Games Dynamics," with H.P. Young, Journal of Theoretical Population Biology, 38, (1990), 219 - 232.
"Cooperation in the Short and in the Long Run," with H.P. Young, Games and Economic Behavior, 3, (1991), 145 - 156.

Other work in "progress"

One current project that has web pointers that I am working on is computer go. We are trying to use pattern matching to develop a go player.

I've currently only put up only one open source project called baby-lakos). It implements a way of doing levilization ala lakos. This was enough to get me certified as a apprentice on Advogato. Hopefully when the go progrect goes open-source I'll upgrade to journeyman.

Another project is the life calculator. Choong Tze Chua is now working on this project with us. He has developed a more complete version of the life calculator.