Mini-course taught at Duke 2021 Fall
Overview
Kernel methods are fundamental tools in machine learning (ML) and high-dimensional data analysis. This course provides a mathematical introduction to the topic to help gain an understanding of the theory and applications of kernel methods, with considerations of computation and real-world applications. The mini-course consists of 9 lectures, starting from an elementary review of reproducing kernel Hilbert space and classical kernel methods in supervised and unsupervised learning. We then go to several research topics using kernels, primarily on high dimensional data: graph-based unsupervised learning (graph Laplacian), Maximum mean discrepancy and two-sample testing, predictive and generative models by neural networks. The second half of the lecture connects kernels to modern neural networks (NN), where we cover some topics in NN approximation, optimization, and applications, where kernel methods play a role. The course goes from preliminary concepts to recent research problems in the field, featuring tentative guest lectures and in-class discussions on selected topics. The course is suitable for graduate students in applied math, statistics, computer science, and electrical engineering, as well as undergraduates with related research backgrounds. Audition welcome and registration encouraged.
Tentative schedule
L1 (9/23): Overview of kernel methods and ML basics
L2 (9/28): Reproducing kernel Hilbert space and function smoothness
L3 (9/30): Kernel regression and kernel PCA
L4 (10/7): Unsupervised learning with kernelized affinity on graphs and manifolds
L5 (10/12): Maximum mean discrepancy and kernel two-sample test
L6 (10/14): Introduction to neural networks and generative neural networks
L7 (10/19): Neural network approximation and kernel (guest lecture)
L8 (10/21): Point process data and influence kernel (guest lecture)
L9 (10/26): Neural network optimization and neural tangent kernel
Course Material
Lecture notes will be distributed as the class goes. List of bibliography and suggested readings will be provided.
Prerequisites
Analysis (multivariate calculus), Linear algebra, Probability. Background in measure theory and functional analysis will be helpful. Experience with coding (e.g. Matlab, python) will be helpful.