2026 (Current Year) Faculty Courses School of Computing Department of Mathematical and Computing Science Graduate major in Mathematical and Computing Science

Statistical Learning Theory

Academic unit or major: Graduate major in Mathematical and Computing Science
Instructor(s): Takafumi Kanamori
Class Format: Lecture (Face-to-face)
Media-enhanced courses: -
Day of week/Period (Classrooms): 7-8 Tue (M-155(H1104)) / 7-8 Fri (M-155(H1104))
Class: -
Course Code: MCS.T403
Number of credits: 200
Course offered: 2026
Offered quarter: 1Q
Syllabus updated: Mar 13, 2026
Language: English

Syllabus

Course overview and goals

Advanced topics in the theory of statistics and machine learning are covered. Specifically, students learn about nonparametric learning methods known as kernel methods, the universal approximation theorem for neural network models, the statistical properties of training and prediction errors, and methods for analyzing generalization error.

Course description and aims

[Objectives] Statistical science and machine learning are disciplines in which useful information is extracted from data to aid
prediction and decision making. Students will learn methodology not simply as knowledge but also learning the background theory including the validity of those methods to promote understanding the essence. Students will broadly apply all kinds of techniques to a variety of problems, learning to construct new techniques on one's own.
[Topics] Students in this course will learn several of statistical science's more advanced techniques, based on their connection to various application fields. We will focus in particular on the connection with machine learning, introducing central topics from both statistical science and machine learning.

Keywords

machine learning, statistics, kernel methods, spline method, neural networks, universal approximation theorem, prediction error, generalization error analysis

Competencies

Specialist skills
Intercultural skills
Communication skills
Critical thinking skills
Practical and/or problem-solving skills

Class flow

The lecture will be conducted using slides. Please bring your laptop, tablet, etc.

Course schedule/Objectives

	Course schedule	Objectives
Class 1	Overview of statistical learning and Regression analysis	Overview the statistical learning through some practical examples. Learn the problem setup of regression analysis.
Class 2	Regression analysis	Understand statistical modeling in kernel regression analysis
Class 3	Regression analysis and kernel methods	Understand cross validation method for regression analysis. Learn statistical inference using kernel method.
Class 4	Kernel methods: reproducing property, representer theorem, etc.	Understand reproducing property, representer theorem, etc used in statistical learning with kernels.
Class 5	Kernel method and reproducing kernel Hilbert space	Understand some properties of reproducing kernel Hilbert space defined from kernel function.
Class 6	Spline smoothing and kernel methods	Learn the relationship between spline smoothing methods and kernel methods.
Class 7	Spline smoothing and kernel methods	Learn B-spline and multi-dimensional spline regression.
Class 8	Neural network models and universal approximation theorem	Understand Neural network models and learn the universal approximation theorem for neural networks.
Class 9	Deep learning models and B-spline method.	Understand deep neural network models and B-spline method.
Class 10	Inequalities of probabilities in machine learning	Understand some probabilistic inequalities used in the theory of machine learning.
Class 11	Foundation of statistical learning theory	Understand the problem setting of statistical learning theory.
Class 12	Prediction error and Model Selection	Learn the prediction error of statistical learning and model selection methods.
Class 13	Generalization error analysis I	Learn generalization error analysis of learning algorithms.
Class 14	Generalization error analysis II	Learn generalization error analysis of learning algorithms.

Study advice (preparation and review)

To enhance effective learning, students are encouraged to spend approximately 100 minutes preparing for class and another 100 minutes reviewing class content afterwards (including assignments) for each class.

Textbook(s)

Unspecified.

Reference books, course materials, etc.

Course materials are provided during the course.
Reference book:
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar, Foundations of Machine Learning, MIT Press, Second Edition, 2018.
Shai Shalev-Shwartz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.

Evaluation methods and criteria

Based on quizzes in class and report submissions

Related courses

MCS.T223 ： Mathematical Statistics
ART.T458 ： Machine Learning

Prerequisites

Students must have basic knowledge of probability theory and statistics.