IE 675b: Machine Learning (HWS 2026)

Machine learning is concerned with building computer systems that improve with experience as well as the study of learning processes, including the design of algorithms that are able to make predictions or extract knowledge from data. The aim of this module is to provide an introduction into the field of machine learning, and study algorithms, underlying concepts, and theoretical principles.

Tentative schedule:

1. Introduction
2. Basics of machine learning (probability refresher, discriminative/generative models, model selection)
3. Inference and decision (estimators, parameter estimation, Bayesian inference, Bayesian decision)
4. Generative models for discrete data (Beta-Binomial, Dirichlet-Multinomial, Naive Bayes)
5. Classifiers for continuous data (logistic regression, softmax regression, Gaussian Naive Bayes)
6. Point estimation (maximum likelihood, empirical risk, gradient descent, stochastic gradient descent, Newton's method, MAP, regularized risk)
7. Dimensionality reduction (matrix decompositions, SVD, PCA, latent linear models)
8. EM algorithm & mixture models
9. Kernels and vector machines (kernel machines, vector machines, kernel trick, sparse vector machines, support vector machines)
10. Hyperparameter optimization (problem, blackbox optimization, multi-fidelty optimization)

Deep learning and more advanced topics are covered in the follow up course “IE 678 Deep Learning”.

In ILIAS. Last year's lecture recordings and lecture slides can be found here.

• K.P. Murphy. Probabilistic Machine Learning: An Introduction. The MIT Press, 2022 (https://probml.github.io/pml-book/book1.html)
• I. Goodfellow, Y. Bengio, A. Courville. Deep Learning. The MIT Press, 2017 (http://www.deeplearningbook.org/)
• Additional material and articles provided in lecture notes