2021 Faculty Courses School of Computing Department of Mathematical and Computing Science Graduate major in Mathematical and Computing Science
Theory of Statistical Mathematics
- Academic unit or major
- Graduate major in Mathematical and Computing Science
- Instructor(s)
- Takafumi Kanamori
- Class Format
- Lecture
- Media-enhanced courses
- -
- Day of week/Period
(Classrooms) - 7-8 Tue / 7-8 Fri
- Class
- -
- Course Code
- MCS.T507
- Number of credits
- 200
- Course offered
- 2021
- Offered quarter
- 1Q
- Syllabus updated
- Jul 10, 2025
- Language
- English
Syllabus
Course overview and goals
Some advanced topics and theories related to statistics and machine learning are taught. More specifically, a nonparametric method called kernel method, statistical properties of training and prediction errors, prediction error bound using Rademacher complexity, are taught.
Course description and aims
[Objectives] Statistical science and machine learning are disciplines in which useful information is extracted from data to aid
prediction and decision making. Students will learn methodology not simply as knowledge but also learning the background theory including the validity of those methods to promote understanding the essence. Students will broadly apply all kinds of techniques to a variety of problems, learning to construct new techniques on one's own.
[Topics] Students in this course will learn several of statistical science's more advanced techniques, based on their connection to various application fields. We will focus in particular on the connection with machine learning, introducing central topics from both statistical science and machine learning.
Keywords
machine learning, statistics, kernel methods, prediction error, Rademacher complexity, statistical consistency
Competencies
- Specialist skills
- Intercultural skills
- Communication skills
- Critical thinking skills
- Practical and/or problem-solving skills
Class flow
Online Lectures using slides.
Course schedule/Objectives
| Course schedule | Objectives | |
|---|---|---|
| Class 1 | Introduction | Introduction of Statistical Mathematics. Understand the problem setup of machine learning through some practical examples. |
| Class 2 | Regression Analysis | Understand the statistical modeling with kernels, regularization, and kernel ridge regressions in regression analysis. |
| Class 3 | Kernel methods I: positive definite kernels | Review positive definite matrix. Understand the definition of positive definite kernels, and learn some properties of kernel functions and examples. |
| Class 4 | Kernel methods II: reproducing property, representer theorem, etc. | Ｌｅａｒ ｔｈｅ ｓｔａｔｉｓｔｉｃａｌ ｍｏｄｅｌｉｎｇ ｗｉｔｈ ｒｅｐｒｏｄｕｃｉｎｇ ｋｅｒｎｅｌ Ｈｉｌｂｅｒｔ ｓｐａｃｅ． |
| Class 5 | Spline smoothing and kernel methods | Learn the relationship between spline smoothing methods and kernel methods. |
| Class 6 | Spline smoothing and kernel methods II | Learn B-spline and multi-dimensional spline regression |
| Class 7 | Classification analysis and kernel methods | Learn kernel-based support vector machine for classification problems |
| Class 8 | Review of Probability theory | Review the probability theory used in Machine Leaning |
| Class 9 | Inequalities in Probability Theory. | Understand some probabilistic inequalities used in machine learning. |
| Class 10 | Problem setup of statistical learning theory | Understand the problem setup of statistical learning theory. Learn the concepts of hypothesis class, training errors, prediction errors, Bayes errors and Bayes rules. |
| Class 11 | Prediction error and Model Selection | Learn the prediction error of statistical learning and model selection methods. |
| Class 12 | Rademacher Complexity | Learn Rademacher complexity to measures the statistical models |
| Class 13 | Uniform law of Large Numbers and Statistical Consistency Learning Algorithms | Learn uniform law of large numbers (ULLN) that is an extension of the law of large numbers. Understand the proof of statistical consistency of learning algorithms. |
| Class 14 | Summary | Summarize Stasistical Learning |
Study advice (preparation and review)
To enhance effective learning, students are encouraged to spend approximately 100 minutes preparing for class and another 100 minutes reviewing class content afterwards (including assignments) for each class.
They should do so by referring to textbooks and other course material.
Textbook(s)
Unspecified.
Reference books, course materials, etc.
Course materials are provided during class.
Reference book: Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.
Evaluation methods and criteria
Evaluated by report submission.
Related courses
- MCS.T223 ： Mathematical Statistics
- MCS.T402 ： Mathematical Optimization: Theory and Algorithms
- MCS.T403 ： Statistical Learning Theory
Prerequisites
It is preferred that students know the basics of statistics and probability theory.