Last modified: Tue Sep 27 12:16:53 EDT 2005 by Dean Foster

Statistical Data mining: High dimensions

First intuition: Always think p > n

Classical statistics has p finite, and n close to infinite.

Short and fat data has p bigger than n. Natural limit is either n fixed and p goes to infinity. Or both go to infinity.

How bad is our intuition about large dimensions?