Computer Vision and Machine Learning
(Prof. Dr.-Ing Margret Keuper)
Our group's research focuses on Computer Vision and Machine Learning. More specifically, we are interested in:
- Robustness and Reliability of Deep Learning Models
- Neural Architecture Search
- Grouping Problems (in applications such as Image and Motion Segmentation and Multiple Object Tracking)
- Efficient Solvers for Large Grouping Problems
- Motion Estimation
- Image Generation and Deep Fake Detection
- Vision for Medical Imaging
- Understanding and building Vision Language Models and other Vision Foundation Models
People
PhD Students
At the University of Mannheim:
Alumni:
- Yumeng Li (Bosch BCAI)
- Patrick Müller
- Amrutha Saseendran (Bosch BCAI)
- Jovita Lukasik
- Amirhossein Kardoost
- Kalun Ho (Fraunhofer ITWM)
External PhD Students:
Teaching
At the University of Mannheim, we are teaching the following courses:
Higher Level Computer Vision (CS 646)
Generative Computer Vision
Master and Bachelor Theses
If you are interested in writing a seminar, Bachelor or Master thesis with us, please feel free to contact us. The list below represents a highly incomplete set of the topics currently offered by our group, talk to us for more topics and additional information. Your own ideas and interests are welcome as well.
- Using uncertainties to improve panoptic segmentation (contact: Shashank)
- Optical Flow NeRFs (contact: Prof. Dr.-Ing Margret Keuper)
- Compressing LLMs for Edge Devices (contact: Shashank)
- Structured Pruning of Vision Models (contact: Shashank)
- Zero Cost Proxies for evaluating Optical Flow Estimation methods (contact: Shashank)
Completed Student Projects
- Team Project: Julian Yuya Caspary, Luca Schwarz and Xinyan Gao: FLOWBENCH: A ROBUSTNESS BENCHMARK FOR OPTICAL FLOW ESTIMATION. November, 2024
- Team Project: Jonas Jakubassa, Simon Kral, and Ruben Weber: DetecBench: A Robustness-Aware Benchmarking Tool For Object Detection. December, 2024
- Team Project: David Schader, Nico Sharei, and Mehmet Ege Kaçar: SemSegBench: Robustness-Aware Benchmarking Of Semantic Segmentation. December, 2024
- Team Project: Amaan Ansari, Annika Dackermann, and Fabian Rösch: DispBench: A Robustness Evaluator For Disparity Estimation. December, 2024
Publications
News:
Y. Li, W. Beluch, M. Keuper, D. Zhang, A. Khoreva: VSTAR: Generative Temporal Nursing for Longer Dynamic Video Synthesis, accepted at ICLR 2025.
P. Gavrikov, J. Lukasik, S. Jung, R. Geirhos, M.J. Mirza, M. Keuper, J. Keuper: Can we talk models into seeing the world differently?, accepted at ICLR 2025.
J. Lukasik, M. Moeller, M. Keuper: An evaluation of zero-cost proxies-from neural architecture performance prediction to model robustness, International Journal of Computer Vision, 1–18, 2025.
K. Prasse, I. Bravo, S. Walter, M. Keuper: I Spy With My Little Eye: A Minimum Cost Multicut Investigation of Dataset Frames, accepted at WACV 2025.
T. Medi, S. Jung, M. Keuper: Fair-TAT: Improving Model Fairness Using Targeted Adversarial Training, arXiv preprint arXiv:2410.23142, accepted at WACV 2025.
S. Agnihotri, J. Grabinski, M. Keuper: Improving Feature Stability during Upsamping -- Spectral Artifacts and the Importance of Spatial Context, ECCV 2024.
P. Gavrikov, S. Agnihotri, M. Keuper, J. Keuper: How Do Training Methods Influence the Utilization of Vision Models?, NeurIPS workshop: Interpretable AI: Past, Present and Future, 2024
K. Bäuerle, P. Müller, S. M. Kazim, I. Ihrke, M. Keuper: Learning the essential in less than 2k additional weights – a simple approach to improve image classification stability under corruptions, [openreview] [pdf] [bib], Transactions on Machine Learning Research (TMLR), 2024.
J. Grabinski, J. Keuper, M. Keuper: As large as it gets – Studying Infinitely Large Convolutions via Neural Implicit Frequency Filters, [openreview] [pdf] [bib] [code], Transactions on Machine Learning Research (TMLR), 2024.
S. Agnihotri, S. Jung, M. Keuper: CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks (https://openreview.net/forum?id=CXZqGJonmt), ICML 2024.
Y. Zhou, M. Fritz, M. Keuper: MultiMax: Sparse and Multi-Modal Attention Learning (https://openreview.net/forum?id=IC9UZ8lm25), ICML 2024.
J. P. Schneider, M. Fatima, J. Lukasik, A. Kolb, M. Keuper, M. Moeller: Implicit Representations for Constrained Image Segmentation (https://openreview.net/forum?id=IaV6AgrTUp), ICML 2024.
U. A. Kaplan, Y. Li, M. Keuper, A. Khoreva, and D. Zhang, “Domain-Aware Fine-Tuning of Foundation Models,” in ICML 2024 Workshop on Foundation Models in the Wild (ICML 2024 FM-Wild Workshop), Vienna, Austria, 2024.
Y. Li, M. Keuper, D. Zhang, A. Khoreva: Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive (https://arxiv.org/abs/2401.08815), ICLR 2024.
J. Lukasik, P. Gavrikov, J. Keuper, M. Keuper: Improving Native CNN Robustness with Filter Frequency Regularization, TMLR 2023.
Y. Li, M. Keuper, D. Zhang, A. Khoreva, Divide & Bind Your Attention for Improved Generative Semantic Nursing, BMVC 2023 (oral).
T. Medi, J. Tayyub, M. Sarmad, F. Lindseth, M. Keuper, FullFormer: Generating Shapes inside Shapes, GCPR 2023.
K. Prasse, S. Jung, Y. Zhou, M. Keuper, Local Spherical Harmonics Improve Skeleton-based Hand Action Recognition, GCPR 2023.
J. Lukasik, J. Geiping, M. Möller, M. Keuper, Differentiable Architecture Search: a One-Shot Method?, AutoML Conference workshop, 2023.
P. Müller, A. Braun, M. Keuper, Classification robustness to common optical aberrations, ICCV2023 AROW Workshop.
S. Agnihotri, K.V. Gandikota, J. Grabinski, P. Chandramouli, M. Keuper, On the unreasonable vulnerability of transformers for image restoration and an easy fix, ICCV2023 AROW Workshop
An Evaluation of Zero-Cost Proxies -- from Neural Architecture Search to Model Robustness, J. Lukasik, M. Möller, M. Keuper, accepted at GCPR, 2023.
An extended Benchmark Study of Human-Like Behavior under Adversarial Training, P. Gavrikov, J. Keuper, M. Keuper, Proceedings of the IEEE/
Improving Primary-Vertex Reconstruction with a Minimum Cost lifted Multicut Graph Partitioning Algorithm, V. Kostyukhin, M. Keuper, I. Ibragimov, N. Owtscharenko, M. Cristinziani, Journal of Instrumentation (JINST) , 2023.
Neural Architecture Design and Robustness: A Dataset, S. Jung, J. Lukasik, M. Keuper, ICLR 2023.
Intra-Source Style Augmentation for Improved Domain Generalization, Y. Li, D. Zhang, M. Keuper, A. Khoreva, WACV 2023.
Trading-off Image Quality for Robustness is not necessary with Regularized Deterministic Autoencoders, A. Saseendran, K. Skubsch, S. Falkner, M. Keuper, NeurIPS 2022.
Robust Models are Less Over-Confident, J. Grabinski, P. Gavrikov, J. Keuper, M. Keuper, NeurIPS 2022.
Learning Where to Look -- Generative NAS is Surprisingly Efficient, J. Lukasik, S. Jung, M. Keuper, ECCV 2022.
FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting, J. Grabinski, S. Jung, J. Keuper, M. Keuper, ECCV 2022.
Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks, S. Jung, M. Keuper, ECML-PKDD 2022.
Higher-Order Multicuts for Geometric Model Fitting and Motion Segmentation, E. Levinkov*, A. Kardoost*, B. Andres and M. Keuper (*equal contribution), in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2022.3148795.
NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search, J. Siems, L. Zimmer, A. Zela, J. Lukasik, M. Keuper and F. Hutter, ICLR, 2022.
Shape your Space: A Gaussian Mixture Regularization Approach to Deterministic Autoencoders , A. Saseendran, K. Skubch, S. Falkner, M. Keuper, NeurIPS 2021.
Spectral Distribution aware Image Generation, S. Jung, M. Keuper, AAAI 2021.
Estimating the Robustness of Classification Models by the Structure of the Learned Feature-Space, K. Ho, F.-J. Pfreundt, J Keuper, M Keuper, https://arxiv.org/abs/2106.12303
Beyond the Spectrum: Detecting Deepfakes by Image Re-Synthesis, Y He, N Yu, M Keuper, M Fritz, accepted at IJCAI 2021
Uncertainty in Minimum Cost Multicuts for Image and Motion Segmentation, A Kardoost, M Keuper, arXiv preprint arXiv:2105.07469, accepted at UAI 2021
Multi-Class Multi-Instance Count Conditioned Adversarial Image Generation, A Saseendran, K Skubch, M Keuper, arXiv preprint arXiv:2103.16795
SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain, P Harder, FJ Pfreundt, M Keuper, J Keuper, arXiv preprint arXiv:2103.03000, accepted at IJCNN 2021
Learning embeddings for image clustering: An empirical study of triplet loss approaches, K Ho, J Keuper, FJ Pfreundt, M Keuper, 2020 25th International Conference on Pattern Recognition (ICPR), 87–94
Object Segmentation Tracking from Generic Video Cues, A Kardoost, S Müller, J Weickert, M Keuper, 2020 25th International Conference on Pattern Recognition (ICPR), 623–630
Smooth variational graph embeddings for efficient neural architecture search, J Lukasik, D Friede, A Zela, F Hutter, M Keuper, arXiv preprint arXiv:2010.04683, accepted at IJCNN 2021
You can also find a full list of my publications on Google Scholar.
- Schäfer, B., Keuper, M. and Stuckenschmidt, H. (2021). Arrow R-CNN for handwritten diagram recognition. International Journal on Document Analysis and Recognition : IJDAR, 24, 3–17.
- Keuper, M., Tang, S., Andres, B., Brox, T. and Schiele, B. (2020). Motion segmentation & multiple object tracking by correlation co-clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 140–153.
- Ilg, E., Saikia, T., Keuper, M. and Brox, T. (2018). Occlusions, motion and depth boundaries with a generic network for disparity, optical flow or scene flow estimation. Computing Research Repository (CoRR), 2018, 1–26.
- He, Y., Keuper, M., Schiele, B. and Fritz, M. (2017). Learning dilation factors for semantic segmentation of street scenes. Computing Research Repository (CoRR), 2017, 1–11.
- Keuper, M. (2017). Higher-order minimum cost lifted multicuts for motion segmentation. Computing Research Repository (CoRR), 2017, 1–11.
- Wannenwetsch, A. S., Keuper, M. and Roth, S. (2017). ProbFlow: Joint optical flow and uncertainty estimation. Computing Research Repository (CoRR), 2017, 1–18.
- Agnihotri, S., Grabinski, J. and Keuper, M. (2025). Improving feature stability during upsampling : spectral artifacts and the importance of spatial context. In , Computer Vision – ECCV 2024 : 18th European Conference, Milan, Italy, September 29-October 4, 2024, proceedings, Part LVIII (S. 357–376). Lecture Notes in Computer Science, Springer: Berlin [u. a.].
- Agnihotri, S., Jung, S. and Keuper, M. (2024). CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks. In , International Conference on Machine Learning, 21–27 July 2024, Viena, Austria (S. 416–451). Proceedings of Machine Learning Research : PMLR, Curran Associates, Inc.: Red Hook, NY.
- Gavrikov, P., Agnihotri, S., Keuper, M. and Keuper, J. (2024). How do training methods influence the utilization of vision models? In , Interpretable AI: past, present and future, IAI Workshop Q NeurIPS 2024, Vancouver, British Columbia, Canada, Dec 15 2024 (S. 1–13). , OpenReview.net: .
- Prasse, K., Jung, S., Zhou, Y. and Keuper, M. (2024). Local spherical harmonics improve skeleton-based hand action recognition. In , Pattern recognition : 45th DAGM German Conference, DAGM GCPR 2023, Heidelberg, Germany, September 19–22, 2023 ; proceedings (S. 67–82). Lecture Notes in Computer Science, Springer: Berlin [u. a.].
- Sommerhoff, H., Agnihotri, S., Saleh, M., Möller, M., Keuper, M., Choubey, B. and Kolb, A. (2024). Task driven sensor layouts – Joint optimization of pixel layout and network parameters. In , 2024 IEEE International Conference on Computational Photography (ICCP) : July 22 2024 to July 24 2024, Lausanne, Switzerland (S. 1–10). , IEEE: Piscataway, NJ.
- Prasse, K., Jung, S., Bravo, I., Walter, S. and Keuper, M. (2023). Towards understanding climate change perceptions : a social media dataset. In , NeurIPS 2023 Workshop on Tackling Climate Change with Machine Learning: Blending New and Existing Knowledge Systems : workshop (S. 1–20). , : .
- Primpeli, A., Bizer, C. and Keuper, M. (2020). Unsupervised bootstrapping of active learning for entity resolution. In , The Semantic Web : 17th International Conference, ESWC 2020, Heraklion, Crete, Greece, May 31-June 4, 2020, Proceedings (S. 215–231). Lecture Notes in Computer Science, Springer: Berlin [u. a.].
- Kardoost, A. and Keuper, M. (2019). Solving minimum cost lifted multicut problems by node agglomeration. In , Computer Vision – ACCV 2018 : 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part IV (S. 74–89). Lecture Notes in Computer Science, Springer: Berlin [u. a.].
- Broscheit, S., Gemulla, R. and Keuper, M. (2018). Learning distributional token representations from visual features. In , ACL 2018, Representation Learning for NLP : Proceedings of the Third Workshop : July 20, 2018 Melbourne, Australia (S. 187–194). , Association for Computational Linguistics: Stroudsburg, PA.
- Ilg, E., Saikia, T., Keuper, M. and Brox, T. (2018). Occlusions, motion and depth boundaries with a generic network for disparity, optical flow or scene flow estimation. In , Computer Vision – ECCV 2018 : 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part XII (S. 626–643). Lecture Notes in Computer Science, Springer: Berlin [u. a.].
- He, Y., Chiu, W.-C., Keuper, M. and Fritz, M. (2017). STD2P: RGBD semantic segmentation using spatio-temporal data-driven pooling. In , 30th IEEE Conference on Computer Vision and Pattern Recognition : CVPR 2017 : 21–26 July 2016, Honolulu, Hawaii : proceedings (S. 7158-7167). , IEEE: Piscataway, NJ.
- He, Y., Keuper, M., Schiele, B. and Fritz, M. (2017). Learning dilation factors for semantic segmentation of street scenes. In , Pattern Recognition : 39th German Conference, GCPR 2017, Basel, Switzerland, September 12–15, 2017, Proceedings (S. 41–51). Lecture Notes in Computer Science, Springer: Berlin [u. a.].
- Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A. and Brox, T. (2017). FlowNet 2.0: Evolution of optical flow estimation with deep networks. In , 30th IEEE Conference on Computer Vision and Pattern Recognition : CVPR 2017 : 21–26 July 2016, Honolulu, Hawaii : proceedings (S. 1647-1655). , IEEE: Piscataway, NJ.
- Keuper, M. (2017). Higher-order minimum cost lifted multicuts for motion segmentation. In , 2017 IEEE International Conference on Computer Vision : ICCV 2017 : proceedings : 22 – 29 October 2017, Venice, Italy (S. 4252 -4260). , IEEE: Piscataway, NJ.
- Wannenwetsch, A. S., Keuper, M. and Roth, S. (2017). ProbFlow: Joint optical flow and uncertainty estimation. In , 2017 IEEE International Conference on Computer Vision : ICCV 2017 : proceedings : 22 – 29 October 2017, Venice, Italy (S. 1182-1191). , IEEE: Piscataway, NJ.
- Gemulla, R., Ponzetto, S. P., Bizer, C., Keuper, M. and Stuckenschmidt, H. (eds.) (2018). LWDA 2018 : Proceedings of the Conference “Lernen, Wissen, Daten, Analysen”, Mannheim, Germany, August 22–24, 2018. Aachen, Germany: RWTH Aachen.