Statistics 433: Introduction to stochastic processes

• Description:

  This course is to be a basic introduction to stochastic processes. The primary focus will be on Markov chains both in discrete time and in continuous time. By focusing attention on Markov chain, we can discuss many interesting models (from physics to economics). Topics covered include: stable distributions, birth-death processes, Poisson processes, time reversibility, random walks, Brownian motion and Black-Scholes.

• Prerequisites:
  • Stat 430.
  • or permission of instructor.
• Spring 2004 will be the first time this course is taught.
• Instructor: Dean Foster.
• Book: Introduction to Stochastic modeling, by Taylor and Karlin, 3rd edition.
• Work load: This course will be driven primarily by problems. Hence there will be problem sets due weekly.
  • Homework = 30% (possibly including some computing / simulations)
  • Midterm = 30%
  • Final = 40%

(or follow links from Foster's home page)

• Time allocation:
  • Chapters 1 & 2: Review of probability (1 weeks)
  • Chapter 3: Markov chains (5 weeks)
  • Chapter 4: Long run behavior of Markov chains (2 weeks)
  • Chapter 5: Poisson processes (3 weeks)
  • Chapter 6: Continuous time Markov chains (3 weeks)
  • Chapter 8: Brownian motion (as time permits)

Details of topics to be covered:

1. A brief review of probability
   1. What is probability?
   2. Rules of probability
   3. Conditional probability
   4. Independence
   5. Some urn models and useful probability models
   6. Random variables, random vector and random processes
   7. Some useful random variables and processes
   8. Expectations
   9. Joint distributions
   10. Moment generating, characteristic, and probability generating functions
   11. Applications and examples
2. A brief review of some useful inequalities and limiting theorems
   1. Markov's inequality
   2. Chebyshev's inequality
   3. Law of large numbers
   4. Central limit theorem
   5. Applications and examples
3. Conditional probability and conditional expectation
   1. Discrete case
   2. Continuous case
   3. Computing expectations by conditioning
   4. Computing probabilities by conditioning
   5. Applications and examples
4. Markov chains
   1. Definition
   2. Initial distribution and transition probability
   3. Markov chains having two states
   4. Applications and examples
   5. Computations with transition probabilities
   6. Hitting times
   7. Chapman-Kolmogorov equations
   8. Classification of states
   9. Induced martingales
   10. Birth and death chains
   11. Limiting and stationary probabilities of Markov chains
   12. More applications and examples
   13. Some related topics
5. Exponential distribution and Poisson process
   1. Properties of exponential distribution
   2. Counting process
   3. Poisson process
   4. Inter-arrival and waiting time distributions
   5. Further properties of Poisson process
   6. Related topics (non-homogeneous, compound processes,...)
   7. Applications and examples
6. Continuous-time Markov chains
   1. Definition
   2. Birth and death process
   3. The Kolmogorov differential equations
   4. Limiting probabilities
   5. Time reversibility
   6. Computing the transition probabilities
   7. Applications and examples
7. Brownian motion
   1. Random walks and Brownian motion
   2. Geometric Brownian Motion
   3. Black-Scholes Option Pricing Formula