Machine Learning and Computer Vision

Our group's research focuses on Computer Vision and Machine Learning. More specifically, we are interested in:

• Robustness and Reliability of Deep Learning Models
• Neural Architecture Search
• Grouping Problems (in applications such as Image and Motion Segmentation and Multiple Object Tracking)
• Efficient Solvers for Large Grouping Problems
• Motion Estimation
• Image Generation and Deep Fake Detection
• Vision for Medical Imaging
• Understanding and building Vision Language Models and other Vision Foundation Models

We are teaching the following courses:

Higher Level Computer Vision (CS 646)

Image Processing (CS 647)

Generative Computer Vision (CS 668)

Reinforcement Learning (IE 695)

Seminar: Computer Vision (CS 717)

If you are interested in writing a seminar, Bachelor or Master thesis with us, please feel free to contact us. The list below represents a highly incomplete set of the topics currently offered by our group, talk to us for more topics and additional information. Your own ideas and interests are welcome as well.

• Using uncertainties to improve panoptic segmentation (contact: Shashank)
• Optical Flow NeRFs (contact: Prof. Dr.-Ing Margret Keuper)
• Compressing LLMs for Edge Devices (contact: Shashank​​​​​​​)
• Structured Pruning of Vision Models (contact: Shashank​​​​​​​)
• Zero Cost Proxies for evaluating Optical Flow Estimation methods (contact: Shashank​​​​​​​)

• Team Project: Julian Yuya Caspary, Luca Schwarz and Xinyan Gao: FLOWBENCH: A ROBUSTNESS BENCHMARK FOR OPTICAL FLOW ESTIMATION. November, 2024
• Team Project: Jonas Jakubassa, Simon Kral, and Ruben Weber: DetecBench: A Robustness-Aware Benchmarking Tool For Object Detection. December, 2024
• Team Project: David Schader, Nico Sharei, and Mehmet Ege Kaçar: SemSegBench: Robustness-Aware Benchmarking Of Semantic Segmentation. December, 2024
• Team Project: Amaan Ansari, Annika Dackermann, and Fabian Rösch: DispBench: A Robustness Evaluator For Disparity Estimation. December, 2024

Peer Reviewed Journal and Conference Papers

2025

• An evaluation of zero-cost proxies-from neural architecture performance prediction to model robustness, J. Lukasik, M. Moeller, M. Keuper, International Journal of Computer Vision, 1--18, 2025.
• DCBM: Data-Efficient Visual Concept Bottleneck Models, K. Prasse, P. Knab, S. Marton, C. Bartelt, M. Keuper,  International Conference on Machine Learning (ICML) 2025.
• Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation, Y. Zhou, M. Keuper, M. Fritz, Annual Meeting of the Association for Computational Linguistics (ACL) 2025.
• Segment any Repeated Object, Y. Liu, C. Graf, M. Spies, M. Keuper, IEEE International Conference on Robotics and Automation (ICRA)
• VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis, Y. Li, W. Beluch, M. Keuper, D. Zhang, A. Khoreva, International Conference on Representation Learning (ICLR) 2025.
• Can we talk models into seeing the world differently? P. Gavrikov, J. Lukasik, S. Jung, R. Geirhos, M.J. Mirza, M. Keuper, J. Keuper, International Conference on Representation Learning (ICLR) 2025.
• I Spy With My Little Eye: A Minimum Cost Multicut Investigation of Dataset Frames , K. Prasse, I. Bravo, S. Walter, M. Keuper , Winter Conference on Computer Vision (WACV) 2025.
• Fair-TAT: Improving Model Fairness Using Targeted Adversarial Training, T. Medi, S. Jung, M. Keuper, Winter Conference on Computer Vision (WACV) 2025.
• DispBench: Benchmarking Disparity Estimation to Synthetic Corruptions, S. Agnihotri, A. Ansari, A. Dackermann, F. Rösch, M. Keuper, Synthetic Data for Computer Vision Workshop@ CVPR 2025
• Corner Cases: How Size and Position of Objects Challenge ImageNet-Trained Models, M. Fatima, S. Jung, M. Keuper, Synthetic Data for Computer Vision Workshop@ CVPR 2025
• Are Synthetic Corruptions A Reliable Proxy For Real-World Corruptions?, S. Agnihotri, D. Schader, N. Sharei, M. Keuper, Synthetic Data for Computer Vision Workshop@ CVPR 2025.

2024

• Learning the essential in less than 2k additional weights – a simple approach to improve image classification stability under corruptions, K. Bäuerle, P. Müller, S. M. Kazim, I. Ihrke, and M. Keuper, Transactions on Machine Learning Research, 2024.
• As large as it gets: Learning infinitely large Filters via Neural Implicit Frequency Functions, J. Grabinski, J. Keuper, M. Keuper, Transactions on Machine Learning Research 2024. (featured certification)
• Intra-& extra-source exemplar-based style synthesis for improved domain generalization, Y. Li, D. Zhang, M. Keuper, A. Khoreva, International Journal of Computer Vision, 132 (2), 446–465, 2024.
• Improving Stability during Upsampling-- Spectral Artifacts and the Importance of Spatial Context, S. Agnihotri, J. Grabinski, M. Keuper, European Conference on Computer Vision (ECCV) 2024.
• CosPGD: A unified white-box adversarial attack for pixel-wise prediction tasks, S. Agnihotri, S. Jung, M. Keuper, International Conference on Machine Learning (ICML) 2024.
• MultiMax: Sparse and Multi-Modal Attention Learning, Y. Zhou, M. Fritz, M. Keuper, International Conference on Machine Learning (ICML) 2024.
• Implicit Representations for Constrained Image Segmentation ,J. P. Schneider, M. Fatima, J. Lukasik, A. Kolb, M. Keuper, M. Moeller, International Conference on Machine Learning (ICML) 2024.
• Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive, Y. Li, M. Keuper, D. Zhang, A. Khoreva, International Conference on Representation Learning (ICLR) 2024.
• Differentiable Sensor Layouts for End-to-End Learning of Task-Specific Camera Parameters, H. Sommerhoff, S. Agnihotri, M. Saleh, M. Moeller, M. Keuper, A. Kolb, ICCP 2024.

2023

• Improving primary-vertex reconstruction with a minimum-cost lifted multicut graph partitioning algorithm, V. Kostyukhin, M. Keuper, I. Ibragimov, N. Owtscharenko, M. Cristinziani, Journal of Instrumentation (JINST), Volume 18, July 2023.
• Higher-Order Multicuts for Geometric Model Fitting and Motion Segmentation, E. Levinkov*, A. Kardoost*, B. Andres and M. Keuper (*equal contribution), IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 45, no. 1, pp. 608–622, 1 Jan. 2023.
• Neural Architecture Design and Robustness: A Dataset, S. Jung, J. Lukasik, M. Keuper, International Conference on Representation Learning (ICLR), 2023.
• Divide & Bind Your Attention for Improved Generative Semantic Nursing, Y. Li, M. Keuper, D. Zhang, A. Khoreva, British Machine Vision Conference (BMVC), 2023 (oral).
• Intra-Source Style Augmentation for Improved Domain Generalization, Y. Li, D. Zhang, A. Khoreva, M. Keuper, Winter Conference on Computer Vision (WACV) 2023.
• Towards Understanding Climate Change Perceptions: A Social Media Dataset, K. Prasse, S. Jung, I. Bravo, S. Walter, M. Keuper, NeurIPS 2023 Workshop on Tackling Climate Change with Machine Learning, 2023.
• Implicit Representations for Image Segmentation, J.-P. Schneider, M. Fatima, J. Lukasik, A. Kolb, M. Keuper, M. Moeller, Neurips 2023 UniReps workshop, 2023.
• Classification robustness to common optical aberrations, P. Müller, A. Braun, M. Keuper, ICCV2023 AROW Workshop, 2023.
• S. Agnihotri, K.V. Gandikota, J. Grabinski, P. Chandramouli, M. Keuper, On the unreasonable vulnerability of transformers for image restoration and an easy fix, ICCV2023 AROW Workshop, 2023.
• An extended Benchmark Study of Human-Like Behavior under Adversarial Training, P. Gavrikov, J. Keuper, M. Keuper, Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023.
• Differentiable Architecture Search: a One-Shot Method? , J. Lukasik, J. Geiping, M. Möller, M. Keuper, AutoML Conference workshop, 2023.
• FullFormer: Generating Shapes inside Shapes, T. Medi, J. Tayyub, M. Sarmad, F. Lindseth, M. Keuper, German Conference on Pattern Recognition (GCPR) 2023.
• Local Spherical Harmonics Improve Skeleton-based Hand Action Recognition, K. Prasse, S. Jung, Y. Zhou, M. Keuper, German Conference on Pattern Recognition (GCPR) 2023.
• An Evaluation of Zero-Cost Proxies -- from Neural Architecture Search to Model Robustness, J. Lukasik, M. Möller, M. Keuper, German Conference on Pattern Recognition (GCPR) 2023.

2022

• Aliasing and adversarial robust generalization of CNNs, J. Grabinski, J. Keuper, M. Keuper, Machine Learning, Springer 2022.
• Trading-off Image Quality for Robustness is not necessary with Deterministic Autoencoders, A. Saseendran, K. Skubsch, M. Keuper, Advances in Neural Information Processing Systems (NeurIPS), 2022 (spotlight).
• Robust Models are Less Over-Confident, J. Grabinski, P. Gavrikov, J. Keuper, M. Keuper, Advances in Neural Information Processing Systems (NeurIPS), 2022.
• Learning Where to Look -- Generative NAS is Surprisingly Efficient, J. Lukasik, S. Jung, M. Keuper, European Conference on Computer Vision (ECCV), 2022.
• FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting, J. Grabinski, S. Jung, J. Keuper, M. Keuper, European Conference on Computer Vision (ECCV), 2022.
• Surrogate NAS Benchmarks: Going beyond the limited search space of tabular NAS benchmarks, J. Siems, L. Zimmer, A. Zela, J. Lukasik, M. Keuper and F. Hutter, International Conference on Learning Representations (ICLR), 2022.
• Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks, S. Jung, M. Keuper, ECML-PKDD 2022.
• Estimating the Robustness of Classification Models by the Structure of the Learned Feature-Space, K. Ho, F.J. Pfreundt, J. Keuper, M. Keuper, AAAI-22 Workshop on Adversarial Machine Learning and Beyond, 2022.
• SP-VIT: Learning 2D Spatial Priors for Vision Transformers , Y. Zhou, W. Xiang, C. Li, B. Wang, X. Wei, L. Zhang, M. Keuper, X Hua, British Machine Vision Conference (BMVC), 2022.
• Optimizing Edge Detection for Image Segmentation with Multicut Penalties, S. Jung, S. Ziegler, A. Kardoost, M. Keuper, German Conference on Pattern Recognition (GCPR), 2022.
• Aliasing coincides with CNNs vulnerability towards adversarial attacks, J. Grabinski, J. Keuper, M. Keuper, AAAI-22 Workshop on Adversarial Learning and Beyond, 2022.
• Is RobustBench/AutoAttack a suitable Benchmark for Adversarial Robustness?, P. Lorenz, D. Straßel, M. Keuper, J. Keuper, AAAI-22 Workshop on Adversarial Learning and Beyond, 2022.

2021

• Arrow R-CNN for Handwritten Diagram Recognition, B. Schäfer, M. Keuper, H. Stuckenschmidt, International Journal on Document Analysis and Recognition (IJDAR) 24(1), 3–17, Springer Berlin Heidelberg 2021.
• Shape your Space: A Gaussian Mixture Regularization Approach to Deterministic Autoencoders, A. Saseendran, K. Skubch, S. Falkner, M. Keuper, Advances in Neural Information Processing Systems (NeurIPS), 2021.
• Multi-Class Multi-Instance Count Conditioned Adversarial Image Generation, A. Saseendran, K. Skubsch, M. Keuper, International Conference on Computer Vision (ICCV), 2021.
• Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis, Y. He, N. Yu, M. Keuper and M. Fritz, International Joint Conference on Artificial Intelligence (IJCAI), 2021.
• Uncertainty in Minimum Cost Multicuts for Image and Motion Segmentation, A. Kardosst and M. Keuper, Uncertainty in Artificial Intelligence (UAI), 2021.
• Spectral Distribution Aware Image Generation, S. Jung, M. Keuper, AAAI Conference on Artificial Intelligence 2021. Detecting AutoAttack in the frequency domain
• P Lorenz, P. Harder, D. Straßel, M. Keuper, J. Keuper, ICML 2021 Workshop on Adversarial Machine Learning, 2021.
• DARTS for Inverse Problem: A Study on Hyperparameter Sensitivity, J. Geiping, J. Lukasik, M. Keuper, M. Möller, NeurIPS 2021 workshop on Inverse Problems, 2021.
• Internalized biases in Fréchet inception distances, S. Jung and M. Keuper, NeurIPS 2021 Workshop on Distribution Shifts, 2021.
• Smooth Variational Graph Embeddings for Efficient Neural Architecture Search, J Lukasik, D Friede, A Zela, H Stuckenschmidt, F Hutter, M Keuper, International Joint Conference on Neural Networks (IJCNN), 2021.
• SpectralDefence: Detecting Adversarial Attacks on CNNs in the Fourier Domain, P. Harder, F.J. Pfreundt, M. Keuper, J. Keuper, International Joint Conference on Neural Networks (IJCNN), 2021.
• MSM:Multi-Stage Multicuts for Scalable Image Clustering, K. Ho, A. Chatzimichalilidis, M. Keuper, J. Keuper, International Conference on High Performance Computing, 267–287, 2021.

2020

• Motion Segmentation & Multiple Object Tracking by Correlation Co-Clustering, M. Keuper, S. Tang, B. Andres, T. Brox and B. Schiele, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 42, no. 1, pp. 140–153, 1 Jan. 2020.
• Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions, R. Durall, M. Keuper, F.J. Pfreundt, J. Keuper, Intl. Conference on Computer Vision and Pattern Recognition (CVPR) 2020.
• Self-supervised Sparse to Dense Motion Segmentation, A. Kardoost, K. Ho, P. Ochs, M. Keuper, ACCV 2020, 15th Asian Conference on Computer Vision, 2020.
• A Two-Stage Minimum Cost Multicut Approach to Self-Supervised Multiple Person Tracking, K. Ho, A. Kardoost, F.-J. Pfreundt, J. Keuper, M. Keuper, ACCV 2020, 15th Asian Conference on Computer Vision, 2020.
• Unsupervised Bootstrapping of Active Learning for Entity Resolution, A. Primpeli, C. Bizer and M. Keuper, European Semantic Web Conference (ESWC), 2020.
• Object Segmentation Tracking from Generic Video Cues, A. Kardoost, S. Müller, J. Weickert and M. Keuper, ICPR 2020, 25th International Conference on Pattern Recognition, 2020.
• Learning Embeddings for Image Clustering: An Empirical Study of Triplet Loss Approaches, K. Ho, J. Keuper, F.J. Pfreundt, M. Keuper, ICPR 2020, 25th International Conference on Pattern Recognition, 2020.
• Neural Architecture Performance Prediction Using Graph Neural Networks, Jovita Lukasik, David Friede, Heiner Stuckenschmidt, M. Keuper, DAGM German Conference on Pattern Recognition (GCPR), 2020.
• A Benders Decomposition Approach to Correlation Clustering, J. Lukasik, M. Keuper, M. Singh, J. Yarkony, SC20 workshop on Machine Learning in High Performance Computing Environments (MLHPC), 2020.

2018

• Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimation, E. Ilg, T. Saikia, M. Keuper, T. Brox, European Conference on Computer Vision (ECCV), 2018.
• Solving Minimum Cost Lifted Multicut Problems by Node Agglomeration, A. Kardoost and M. Keuper, ACCV 2018, 14th Asian Conference on Computer Vision, 2018.
• Learning Distributional Token Representations from Visual Features, S. Broscheit, R. Gemulla, M. Keuper, ACL 2018, Representation Learning for NLP : Proc. of the Third Workshop, Melbourne, Australia (187–194), Association for Computational Linguistics, 2018. 

2017

• Higher-Order Minimum Cost Lifted Multicuts for Motion Segmentation, M. Keuper, International Conference on Computer Vision (ICCV), 2017.
• Joint optical flow and uncertainty estimation, A. S. Wannenwetsch, M. Keuper, and S. Roth, International Conference on Computer Vision (ICCV), 2017.
• STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data Driven Pooling, Y. He, W.-C. Chiu, M. Keuper, M. Fritz , Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
• FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks, E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, T. Brox, Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
• Learning Dilation Factors for Semantic Segmentation of Street Scenes, Y. He, M. Keuper, B. Schiele and M. Fritz , DAGM German Conference on Pattern Recognition (GCPR), 2017. 

2016

• Point-wise mutual information-based video segmentation with high temporal consistency, M. Keuper and T. Brox, In Computer Vision – ECCV 2016 Workshops, LNCS, Springer: Cham., 2016.

2015

• Motion Trajectory Segmentation via Minimum Cost Multicuts, M. Keuper, B. Andres, T. Brox, International Conference on Computer Vision (ICCV), 2015.
• Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts, M. Keuper, E. Levinkov, N. Bonneel, G. Lavoue, T. Brox, B. Andres, International Conference on Computer Vision (ICCV), 2015. 

2014

• Spectral graph reduction for efficient image and streaming video segmentation, F. Galasso, M. Keuper, T. Brox, B. Schiele, Conference on Computer Vision and Pattern Recognition (CVPR), 2014. (oral). 

2013

• Blind deconvolution of widefield fluorescence microscopic data by regularization of the optical transfer function (OTF), M. Keuper, T. Schmidt, M. Temerinac-Ott, J. Padeken, P. Heun, O. Ronneberger, T. Brox, Conference on Computer Vision and Pattern Recognition (CVPR), 2013.