CS 707: Data and Web Science Seminar (HWS 2019)
The Data and Web Science seminar covers recent topics in data and web science. This term, the seminar focuses on Automated Machine Learning (AutoML).
Organization
- This seminar is organized by Kiril Gashteovski and Prof. Dr. Rainer Gemulla.
- Available for Master students (2 SWS, 4 ECTS). If you are a Bachelor student and want to take this seminar (2 SWS, 5 ECTS), please contact Prof. Gemulla.
- Prerequisites: solid background in machine learning
- The maximum number of participants is 10 students
Goals
In this seminar, you will
- Read, understand, and explore scientific literature
- Summarize a current research topic in a concise report (10 single-column pages + references)
- Give two presentations about your topic (3 minutes flash presentation, 15 minutes final presentation)
- Moderate a scientific discussion about the topic of one of your fellow students
- Provide feedback to a report and to a presentation of a fellow student
Schedule
- Register as described below.
- Attend the kickoff meeting on Sep 10.
- Work individually throughout the semester according to the seminar schedule.
- Meet your advisor for guidance and feedback.
Registration
Register via Portal 2 until 2 September.
If you are accepted into the seminar, explore the list of topics below and select at least 4 topics of your preference. You may also propose alternative topics relevant to the seminar. Provide your preferred topics by 8 September via email to Kiril Gashteovski. The actual topic assignment takes place soon afterwards; we will notify you via email. Our goal is to assign to you to one of your preferred topics.
Topics
We provide a list of topics, together with a reference paper per topic. Your presentation and report should explore the topic with an emphasis on the reference paper, but not just the reference paper. You may suggest a topic of your own (this literature overview may be helpful).
- Combined algorithm selection and hyperparameter optimization
Auto-WEKA: Combined Selection and Hyperparameters Optimization of Classification Algorithms
Chris Thornton, Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown
KDD 2013 - Bayesian optimization using neural networks
Scalable Bayesian Optimization Using Deep Neural Networks
Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, Ryan P. Adams
ICML 2015 - Approximate gradient-based methods for hyperparameter optimization
Hyperparameter optimization with approximate gradient
Fabian Pedregosa
ICML 2016 - Bandit-based methods for hyperparameter optimization
Non-stochastic Best Arm Identification and Hyperparameter Optimization
Kevin G Jamieson, Ameet Talwalkar
AISTATS 2016 - Bayesian optimization on large datasets
Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets
Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, Frank Hutter
AISTATS 2017 - Learning curve prediction
Learning curve prediction with Bayesian Neural Networks
Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter
ICLR 2017 - Neural architecture search with reinforcement learning
Neural Architecture Search with Reinforcement Learning
Barret Zoph, Quoc V. Le
ICLR 2017 - Why hyperparameter optimization?
Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
Nils Reimers, Iryna Gurevych
EMNLP 2017 - Bayesian optimization in high-dimensional spaces
Batched Large-scale Bayesian Optimization in High-Dimensional Spaces
Zi Wang, Clement Gehring, Pushmeet Kohli, Stefanie Jegelka
AISTATS 2018 - Combining Bayesian optimization and bandit-based methods
BOHB: Robust and Efficient Hyperparameter Optimization at Scale
Stefan Falkner, Aaron Klein, Frank Hutter
ICML 2018 - Neural architecture search with Bayesian optimization
Neural Architecture Search with Bayesian Optimisation and Optimal Transport
Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabás Póczos, Eric P. Xing
NIPS 2018 - Transfer learning in AutoML
Transfer Learning with Neural AutoML
Catherine Wong, Neil Houlsby, Yifeng Lu, Andrea Gesmundo
NIPS 2018 - Meta-learning using neural networks
A Simple Neural Attentive Meta-Learner
Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel
ICLR 2018 - Low-latency neural architecture search
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, Quoc V. Le
CVPR 2019 - Differentiable architecture search
DARTS: Differentiable architecture search
Hanxiao Liu, Karen Simonyan, Yiming Yang
ICLR 2019 - Interactive AutoML
Democratizing Data Science through Interactive Curation of ML Pipelines
Zeyuan Shang, Emanuel Zgraggen, Benedetto Buratti, Ferdinand Kossmann, Philipp Eichmann, Yeounoh Chung, Carsten Binnig, Eli Upfal, Tim Kraska
SIGMOD 2019
Supplementary materials and references
- “Giving Conference Talks” by Prof. Dr. Rainer Gemulla
- “Writing for Computer Science” by Justin Zobel