Chapter III: Markov chains

Admistrivia

• Questions from the homework?

Chapter III.1: Defns

Markov process

• almost "memory less"
• or better--only short term memory
• P(X_t|X₁,X₂,...X_t-1) = P(X_t|X_t-1)
• Much stronger than martingale since probabilities fully describe the world

Examples

• the market (even stochastic volitility models are Markov)
• good marketing decisions
• quantum mechanics
• populations (both humans and viruses)
• genetics
• Human speach understanding

Transition probability

• matrix of probabilities
• Along with a starting distribution FULLY describes the world (Theorem at end of class)
• Just keep multiplying up to get probably of any sequence

Rows sum to one

Infinite matrixes!?!

• Why? Populations for example are most easilly modeled as being unbounded.
• Difficult to draw--easy to notate
• Natural to consider only a piece of matrix and end up with "sub probabilities."

Theorem: multiplying probabilities

• probability version of proof (concrete as in book)
• proof called first step analysis