2025 (Current Year) Faculty Courses School of Computing Undergraduate major in Mathematical and Computing Science
Markov Analysis
- Academic unit or major
- Undergraduate major in Mathematical and Computing Science
- Instructor(s)
- Naoto Miyoshi / Yumiharu Nakano
- Class Format
- Lecture (Face-to-face)
- Media-enhanced courses
- -
- Day of week/Period
(Classrooms) - 7-8 Tue / 7-8 Fri
- Class
- -
- Course Code
- MCS.T312
- Number of credits
- 200
- Course offered
- 2025
- Offered quarter
- 2Q
- Syllabus updated
- Mar 19, 2025
- Language
- Japanese
Syllabus
Course overview and goals
This course facilitates students in understanding of the fundamentals of Markov processes, one of most basic stochastic processes, through analyses of stochastic models.
Course description and aims
At the end of this course, students will be able to:
1) Have understandings of the concept of Markov property in discrete and continuous time, and the basic facts that hold in Markov processes.
2) Apply the theory of Markov processes to analyze various stochastic models.
Keywords
Markov processes, stochastic models, Markov chains, Poisson processes
Competencies
- Specialist skills
- Intercultural skills
- Communication skills
- Critical thinking skills
- Practical and/or problem-solving skills
Class flow
Slides and blackboards will be used.
Course schedule/Objectives
| Course schedule | Objectives | |
|---|---|---|
| Class 1 | Markov property and discrete time Markov chains | Explain descrete-time Markov chains and their applications. |
| Class 2 | Connectivity and periodicity of Markov chains | Explain the concepts and basic properties of the connectivity and periodicity. |
| Class 3 | Recurrence of Markov chains | Explain the concept and basic properties of the recurrence. |
| Class 4 | Stationary distributions | Explain the concepts of the stationary distributions and invariant measures |
| Class 5 | Existence condition of stationary distributions | Explain the existence condition of stationary distributions. |
| Class 6 | Limit theorems | Explain the limit theorems. |
| Class 7 | Transient Properties | Explain the transient properties. |
| Class 8 | Midterm Assessment | Check the understanding of students so far. |
| Class 9 | Poisson processes | Understand the definition of Poisson processes and explain its basic properties. |
| Class 10 | Continuous time Markov chains | Understand the definition of Markov chains in continuous time and explain its basic properties. |
| Class 11 | Infinitesimal Generators | Explain the infinitesimal generators for continuous-time Markov chains. |
| Class 12 | Kolmogorov's differential equations and regularity | Explain Kolmogorov's differential equations and the notion of regularity. |
| Class 13 | Stationary distributions, reversibility and limit theorems | Explain the stationary distributions, reversibility and limit theorems of continuous-time Markov chains. |
| Class 14 | Birth-Death-type queueing models | Introduce some examples of queueing models represented as birth-death processes. |
Study advice (preparation and review)
To enhance effective learning, students are encouraged to prepare in advance and to review afterwards the content of the class .
Textbook(s)
Lecture slides
Reference books, course materials, etc.
P. Brémaud, Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues, Springer
Evaluation methods and criteria
Students will be assessed on the understanding of Markov chains and its application. Grades are based on the results of a midterm assessment and a final exam.
Related courses
- MCS.T212 ： Fundamentals of Probability
Prerequisites
It is preferable that students have completed MCS.T212: Fundamentals of Probability.