Advanced Methods in Text Analytics (IE 696)

This MSc course is a follow-up to IE 661 Text Analytics, where basic concepts and methods for natural languange processing (NLP) are introduced. In this course, we dive deep into the latest state-of-the-art methods for NLP. This means this course is heavily focused on language models implemented using deep learning architectures. Among other things, we will take a close look at the transformer architecture and its applications, including the design, training and applications of large language models (LLMs), the technology behind products like ChatGPT. Much of the fundamentals of the course are additionally supported with hands-on coding exercises using Python.

This course is not focused on how to use LLMs, but on how LLMs work internally. This is crucial for an effective use of LLMs, so even the hands-on exercises are not designed to give practical experience with LLM applications, but with the training and evaluation of LLMs. If you are interested in learning how to use LLMs in applied settings, consider this course instead: Large Language Models and Agents.

Prerequisites

• BSc-level linear algebra and probability theory, as well as a bit of calculus.
• It is strongly recommended to have passed one of the following courses: Machine Learning or Deep Learning 

Time and Location

• Lectures: Tuesdays 13:45 – 15:15 in A5 6 Room C014
• Tutorials: Wednesdays 08:30–10:00 in A5 6 Room C014

Grading / Evaluation

• 100% final exam

Instructors

• Dr. Daniel Ruffinelli
• Prof. Dr. Simone Paolo Ponzetto

Course materials and up-to-date information can be found in our ILIAS group.

Most of the advanced material comes from recent research papers referenced in the course materials, but the basics are based on the following textbooks:

• Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition, Dan Jurafsky and James H. Martin, 3rd Edition, 2026 (author draft here)
• Natural Language Processing, Yue Zhang and Zhiyang Teng, 2021.
• Natural Language Processing, Jacob Eisenstein, 2018.