Admistrivia

• Readings: Chapter 3 up to page 126
• Questions from the homework?

Review example

• Prob of starting in 0 and ending in 1???

General matrix

• Put absorbing states last
• Hence the zeros are all together

  Q	R
  0	I

• First few states are transiant
• Last bunch are absorbing.
• Can work out probability of absorbing into state k
• Matrix of answers! (starting state, ending state)
• U_ik = P_ik + sum P_ij U_jk

Chapter III.5: Examples

2 state limit

• P =	1-a a b 1-b

• L =

  1 a 1 -b

• D =	1 0 0 1-a-b

• R =

  b a 1 -1

  /(a+b)
• CHECK: RL = I
• Check: P = LDR
• Now multiply Pⁿ by P to generate Pⁿ⁺¹.
• Inductive argument done carefully...
• Now do it directly

Random walk definitions

• Big matrix! (q = move left, p = move right, r = stay put)
• State dependent (q_i, p_i, r_i)
• Top and bottom: reflecting or absorbing?
• Good model of betting with my big brother (odds changed based on who was ahead. We both liked gambling.)
• State independent: q, 0, p.