2023 Faculty Courses School of Computing Undergraduate major in Mathematical and Computing Science
Markov Analysis
- Academic unit or major
- Undergraduate major in Mathematical and Computing Science
- Instructor(s)
- Yumiharu Nakano / Naoto Miyoshi
- Class Format
- Lecture (Face-to-face)
- Media-enhanced courses
- -
- Day of week/Period
(Classrooms) - 7-8 Tue (W8E-307(W833)) / 7-8 Fri (W8E-307(W833))
- Class
- -
- Course Code
- MCS.T312
- Number of credits
- 200
- Course offered
- 2023
- Offered quarter
- 2Q
- Syllabus updated
- Jul 8, 2025
- Language
- Japanese
Syllabus
Course overview and goals
This course facilitates students in understanding of the fundamentals of Markov processes, one of most basic stochastic processes, through analyses of stochastic models.
Course description and aims
At the end of this course, students will be able to:
1) Have understandings of the concept of Markov property in discrete and continuous time, and the basic facts that hold in Markov processes.
2) Apply the theory of Markov processes to analyze various stochastic models.
Keywords
Markov processes, stochastic models, Markov chains, Poisson processes
Competencies
- Specialist skills
- Intercultural skills
- Communication skills
- Critical thinking skills
- Practical and/or problem-solving skills
Class flow
Slides and blackboards will be used.
Course schedule/Objectives
| Course schedule | Objectives | |
|---|---|---|
| Class 1 | Markov property and discrete time Markov chains | Explain the concept of Markov properties. |
| Class 2 | Transition diagram and probability distributions of the state | Explain the transition diagram and probability distribution of the state. |
| Class 3 | Classification of the state: connectivity | Classificate the state of Markov chains. |
| Class 4 | Periodicity | Explain the concept and basic facts of the periodicity. |
| Class 5 | Recurrence | Explain the concept and basic facts of the recurrence. |
| Class 6 | Stationary distributions | Explain the concept of the stationary distributions and its derivation. |
| Class 7 | Limit theorems | Explain the limit theorems. |
| Class 8 | Markov chain Monte Carlo methods | Introduce Markov chain Monte Carlo methods. |
| Class 9 | Poisson processes | Understand the definition of Poisson processes and explain its basic properties. |
| Class 10 | Compound Poisson processes | Understand the definition of compound Poisson processes and explain its basic properties. |
| Class 11 | Continuous time Markov chains | Understand the definition of Markov chains in continuous time and explain its basic properties. |
| Class 12 | Birth-death processes | Explain the basic properties and applications of birth-death processes. |
| Class 13 | Queueing systems | Explain the basic properties and applications of queueing systems. |
| Class 14 | Brownian motion | Introduction to Brownian motion |
Study advice (preparation and review)
To enhance effective learning, students are encouraged to spend approximately 100 minutes preparing for class and another 100 minutes reviewing class content afterwards (including assignments) for each class.
They should do so by referring to textbooks and other course material.
Textbook(s)
Lecture notes.
Reference books, course materials, etc.
P. Brémaud, Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues, Springer
Evaluation methods and criteria
Students will be assessed on the understanding of Markov processes and its application. Grades are based on exercises and a final exam.
Related courses
- MCS.T212 ： Fundamentals of Probability
Prerequisites
It is preferable that students have completed MCS.T212: Fundamentals of Probability.