CS 707: Data and Web Science Seminar (HWS 2019)

The Data and Web Science seminar covers recent topics in data and web science. This term, the seminar focuses on Automated Machine Learning (AutoML).

Organization

  • This seminar is organized by Kiril Gashteovski and Prof. Dr. Rainer Gemulla.
  • Available for Master students (2 SWS, 4 ECTS). If you are a Bachelor student and want to take this seminar (2 SWS, 5 ECTS), please contact Prof. Gemulla.
  • Prerequisites: solid background in machine learning
  • The maximum number of participants is 10 students

Goals

In this seminar, you will

  • Read, understand, and explore scientific literature
  • Summarize a current research topic in a concise report (10 single-column pages + references)
  • Give two presentations about your topic (3 minutes flash presentation, 15 minutes final presentation)
  • Moderate a scientific discussion about the topic of one of your fellow students
  • Provide feedback to a report and to a presentation of a fellow student

Schedule

  • Register as described below.
  • Attend the kickoff meeting on Sep 10.
  • Work individually throughout the semester according to the seminar schedule.
  • Meet your advisor for guidance and feedback.

Registration

Register via Portal 2 until September 2.

If you are accepted into the seminar, explore the list of topics below and select at least 4 topics of your preference. You may also propose alternative topics relevant to the seminar. Provide your preferred topics by September 8 via email to Kiril Gashteovski. The actual topic assignment takes place soon afterwards; we will notify you via email. Our goal is to assign to you to one of your preferred topics.

Topics

We provide a list of topics, together with a reference paper per topic. Your presentation and report should explore the topic with an emphasis on the reference paper, but not just the reference paper. You may suggest a topic of your own (this literature overview may be helpful).

  1. Combined algorithm selection and hyperparameter optimization
    Auto-WEKA: Combined Selection and Hyperparameters Optimization of Classification Algorithms
    Chris Thornton, Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown
    KDD 2013
  2. Bayesian optimization using neural networks
    Scalable Bayesian Optimization Using Deep Neural Networks
    Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, Ryan P. Adams
    ICML 2015
  3. Approximate gradient-based methods for hyperparameter optimization
    Hyperparameter optimization with approximate gradient
    Fabian Pedregosa
    ICML 2016
  4. Bandit-based methods for hyperparameter optimization
    Non-stochastic Best Arm Identification and Hyperparameter Optimization
    Kevin G Jamieson, Ameet Talwalkar
    AISTATS 2016
  5. Bayesian optimization on large datasets
    Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets
    Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, Frank Hutter
    AISTATS 2017
  6. Learning curve prediction
    Learning curve prediction with Bayesian Neural Networks
    Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, Frank Hutter
    ICLR 2017
  7. Neural architecture search with reinforcement learning
    Neural Architecture Search with Reinforcement Learning
    Barret Zoph, Quoc V. Le
    ICLR 2017
  8. Why hyperparameter optimization?
    Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks
    Nils Reimers, Iryna Gurevych
    EMNLP 2017
  9. Bayesian optimization in high-dimensional spaces
    Batched Large-scale Bayesian Optimization in High-Dimensional Spaces
    Zi Wang, Clement Gehring, Pushmeet Kohli, Stefanie Jegelka
    AISTATS 2018
  10. Combining Bayesian optimization and bandit-based methods
    BOHB: Robust and Efficient Hyperparameter Optimization at Scale
    Stefan Falkner, Aaron Klein, Frank Hutter
    ICML 2018
  11. Neural architecture search with Bayesian optimization
    Neural Architecture Search with Bayesian Optimisation and Optimal Transport
    Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabás Póczos, Eric P. Xing
    NIPS 2018
  12. Transfer learning in AutoML
    Transfer Learning with Neural AutoML
    Catherine Wong, Neil Houlsby, Yifeng Lu, Andrea Gesmundo
    NIPS 2018
  13. Meta-learning using neural networks
    A Simple Neural Attentive Meta-Learner
    Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel
    ICLR 2018
  14. Low-latency neural architecture search
    MnasNet: Platform-Aware Neural Architecture Search for Mobile
    Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, Quoc V. Le
    CVPR 2019
  15. Differentiable architecture search
    DARTS: Differentiable architecture search
    Hanxiao Liu, Karen Simonyan, Yiming Yang
    ICLR 2019
  16. Interactive AutoML
    Democratizing Data Science through Interactive Curation of ML Pipelines
    Zeyuan Shang, Emanuel Zgraggen, Benedetto Buratti, Ferdinand Kossmann, Philipp Eichmann, Yeounoh Chung, Carsten Binnig, Eli Upfal, Tim Kraska
    SIGMOD 2019

Supplementary materials and references