Statistical analysis of Linguistic data

This is a special topics course on linguistic data. More and more data these days have linguistic content--so this class will investigate what it takes to drop such linguistic data into a statistical model.

The main readings will be:

Speech and Language processing by Jurafsky and Martin. (main text)
Foundations of statistical natural language processing by Manning and Schutze. (Suggested if you have a weaker statistics background.)
Language log (In particular, posts by Mark Liberman often have a lot of statistical content.)

Homework:

I've posted the homework from pervious years (.pdf, .Rnw and for a quick .html view). I'll update this file as the semester goes along. So don't print it out! Just keep checking the web.

Homework 1 is due Sept 24th.

Schedule:

Sept 5: Introduction (.pdf)

Sept 10: Regular expressions (.pdf)
- Chapter 2 of JM,
- chapter 2 and chapter 4 of the O'Reilly book on Regular Expressions.
Sept 12: Ngrams (.pdf)
- N-grams (Chapter 4 of JM)
Sept 13 at noon: Justin Rising and Josh Magarick are running a session called "Python for Statisticians". Lunch will be served!

Sept 19: High diminsional reasoning
- Read before class:
  - JL theorem
  - Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions
Sept 21: Lyle on eigenwords (in the old room SHDH 105)

Sept 24: Speech: encoding (.pdf) (chapters 7 and 8)
- NEW ROOM! R2D2 441
- Limits on D to A: Shannon limit
- Limits on A to D: Nyquist rate
- IPA (fancy alphabet: read along as various people speak words.)
- Homework 1 is due
- the regular expression as a single file
Sept 26: Speech: decoding (chapters 9 and 10)

Oct 1: Backoff and information theory.
Oct 3:

Oct 8: Jordan: guest lecture on HMM's
- Homework 2 due date
Oct 10: Adi on variable length markov chains

Oct 15: Context Free Grammars
- .pdf
- read chapters 12 and 16.of JM
Oct 17:

Oct 22: Homework 3 due
Oct 24: Risk inflation
- Risk inflation paper
- Implementation example (bankruptcy)

Oct 29: No class: rain day
Oct 31: Streaming methods
- lecture notes
- Papers
  - A good introduction to the ideas of risk ratios (Dongyu Lin)
  - Bankruptcy example (67000 regressors)
  - mFDR (with martingale proof)
  - the martingale

Nov 5: The power of large blocks
- risk graphic (finishing last weeks class)
- slides for Dongyu's job talk
Nov 7: Parsing
- .pdf
- read chapters 13 and 14 of JM.

Nov 12:Statistical parsing
- dependency parsing
- ``Parameter Estimation for Statistical Parsing Models: Theory and Practice of Distribution-Free Methods'' by Michael Collins.
Nov 14: notes

Nov 26: Machine traslation (chapter 25)
- Read chapters 17, 18 and 25
- nice tutorial on IBM models 1,2,3, by Kevin Knight
Nov 28: CCA
- slides for today's lecture
- paper with Sham
- CCA goes back to the 1930's, so there should be pleanty of web material to look over. I won't put it up. But if you find something nice, email it to me and I'll post it.

Dec 3: Disambiguation
- Other disambiguation solutions
- Read chapter 20
Dec 5: Hadamard transformations?

dean.foster@gmail.com

Last modified: Wed Dec 5 14:23:59 EST 2012