CS 707: Data and Web Science Seminar (HWS 2019)

The Data and Web Science seminar covers recent topics in data and web science. This term, the seminar focuses on Automated Machine Learning (AutoML).

• This seminar is organized by Kiril Gashteovski and Prof. Dr. Rainer Gemulla.
• Available for Master students (2 SWS, 4 ECTS). If you are a Bachelor student and want to take this seminar (2 SWS, 5 ECTS), please contact Prof. Gemulla.
• Prerequisites: solid background in machine learning
• The maximum number of participants is 10 students

In this seminar, you will

• Read, understand, and explore scientific literature
• Summarize a current research topic in a concise report (10 single-column pages + references)
• Give two presentations about your topic (3 minutes flash presentation, 15 minutes final presentation)
• Moderate a scientific discussion about the topic of one of your fellow students
• Provide feedback to a report and to a presentation of a fellow student

• Register as described below.
• Attend the kickoff meeting on Sep 10.
• Work individually throughout the semester according to the seminar schedule.
• Meet your advisor for guidance and feedback.

Register via Portal 2 until 2 September.

If you are accepted into the seminar, explore the list of topics below and select at least 4 topics of your preference. You may also propose alternative topics relevant to the seminar. Provide your preferred topics by 8 September via email to Kiril Gashteovski. The actual topic assignment takes place soon afterwards; we will notify you via email. Our goal is to assign to you to one of your preferred topics.

We provide a list of topics, together with a reference paper per topic. Your presentation and report should explore the topic with an emphasis on the reference paper, but not just the reference paper. You may suggest a topic of your own (this literature overview may be helpful).

1. Combined algorithm selection and hyperparameter optimization
   Auto-WEKA: Combined Selection and Hyperparameters Optimization of Classification Algorithms
   Chris Thornton, Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown
   KDD 2013
2. Bayesian optimization using neural networks
   Scalable Bayesian Optimization Using Deep Neural Networks
   Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, Ryan P. Adams
   ICML 2015
3. Approximate gradient-based methods for hyperparameter optimization
   Hyperparameter optimization with approximate gradient
   Fabian Pedregosa
   ICML 2016
4. Bandit-based methods for hyperparameter optimization
   Non-stochastic Best Arm Identification and Hyperparameter Optimization
   Kevin G Jamieson, Ameet Talwalkar
   AISTATS 2016
5. Bayesian optimization on large datasets
   Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets
   Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, Frank Hutter
   AISTATS 2017
6. Learning curve prediction
   Learning curve prediction with Bayesian Neural Networks
   Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter
   ICLR 2017
7. Neural architecture search with reinforcement learning
   Neural Architecture Search with Reinforcement Learning
   Barret Zoph, Quoc V. Le
   ICLR 2017
8. Why hyperparameter optimization?
   Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
   Nils Reimers, Iryna Gurevych
   EMNLP 2017
9. Bayesian optimization in high-dimensional spaces
   Batched Large-scale Bayesian Optimization in High-Dimensional Spaces
   Zi Wang, Clement Gehring, Pushmeet Kohli, Stefanie Jegelka
   AISTATS 2018
10. Combining Bayesian optimization and bandit-based methods
    BOHB: Robust and Efficient Hyperparameter Optimization at Scale
    Stefan Falkner, Aaron Klein, Frank Hutter
    ICML 2018
11. Neural architecture search with Bayesian optimization
    Neural Architecture Search with Bayesian Optimisation and Optimal Transport
    Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabás Póczos, Eric P. Xing
    NIPS 2018
12. Transfer learning in AutoML
    Transfer Learning with Neural AutoML
    Catherine Wong, Neil Houlsby, Yifeng Lu, Andrea Gesmundo
    NIPS 2018
13. Meta-learning using neural networks
    A Simple Neural Attentive Meta-Learner
    Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel
    ICLR 2018
14. Low-latency neural architecture search
    MnasNet: Platform-Aware Neural Architecture Search for Mobile
    Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, Quoc V. Le
    CVPR 2019
15. Differentiable architecture search
    DARTS: Differentiable architecture search
    Hanxiao Liu, Karen Simonyan, Yiming Yang
    ICLR 2019
16. Interactive AutoML
    Democratizing Data Science through Interactive Curation of ML Pipelines
    Zeyuan Shang, Emanuel Zgraggen, Benedetto Buratti, Ferdinand Kossmann, Philipp Eichmann, Yeounoh Chung, Carsten Binnig, Eli Upfal, Tim Kraska
    SIGMOD 2019

• “Giving Conference Talks” by Prof. Dr. Rainer Gemulla
• “Writing for Computer Science” by Justin Zobel