CS 707: Data and Web Science Seminar (FSS 2022)

The Data and Web Science seminar covers recent topics in data and web science. The topic for this term is training deep neural networks.

• This seminar is organized by Prof. Dr. Rainer Gemulla and Daniel Ruffinelli.
• Available for Master students (2 SWS, 4 ECTS).
• Prerequisites: solid background in machine learning
• Maximum number of participants is 10 MSc students

In this seminar, you will

• Read, understand, and explore scientific literature
• Summarize a current research topic in a concise report (10 single-column pages + references)
• Give two presentations about your topic (3 minutes flash presentation, 15 minutes final presentation)
• Moderate a scientific discussion about the topic of one of your fellow students
• Review a (draft of a) report of a fellow student

• Register as described below.
• Attend the online kickoff meeting on Feb 23, 17:15 (tentative).
• Work individually throughout the semester according to the seminar schedule (tentative).
• Meet your advisor for guidance and feedback.

Register via Portal 2 until Feb 14.

If you are accepted into the seminar, provide at least 4 topics of your preference (your own and/or example topics; see below) by Feb 20 via email to Daniel Ruffinelli. The actual topic assignment takes place soon afterwards; we will notify you via email. Our goal is to assign one of your preferred topics to you.

Each student works on a topic within the area of the seminar. Your presentation and report should explore the topic with an emphasis on 2 or 3 focus papers. We provide example topics below. If you want, you may suggest a different topic within the area of training deep neural networks (talk to us before the topic assignments). A good starting point is recent research papers in top data mining and machine learning conferences (e.g., try NeurIPS, ICLR, ICML, KDD).

Topic list

1. Optimizers
2. Preventing overfitting and underfitting
3. Activation functions
4. Initialization of network weights
5. Parallel training
6. Federated training
7. Privacy-preserving learning
8. Skip connections for Computer Vision
9. Skip connections for Natural Language Processing
10. Self-supervised training for Computer Vision
11. Self-supervised training for Natural Language Processing
12. Multi-task learning
13. Transfer learning

• “Giving Conference Talks”by Prof. Dr. Rainer Gemulla
• “Writing for Computer Science” by Justin Zobel, Springer, 2014