Computer Vision and Machine Learning

(Prof. Dr.-Ing Margret Keuper)

Our group's research focuses on Computer Vision and Machine Learning. More specifically, we are interested in:

Robustness and Reliability of Deep Learning Models
Neural Architecture Search
Grouping Problems (in applications such as Image and Motion Segmentation and Multiple Object Tracking)
Efficient Solvers for Large Grouping Problems
Motion Estimation
Image Generation and Deep Fake Detection
Vision for Medical Imaging
Understanding and building Vision Language Models and other Vision Foundation Models

People

Chair

Prof. Dr.-Ing Margret Keuper

Office

Anastasia Kamat

PhD Students

At the University of Mannheim:

Alumni:

External PhD Students:

Teaching

At the University of Mannheim, we are teaching the following courses:

Higher Level Computer Vision (CS 646)

Image Processing (CS 647)

Generative Computer Vision

Reinforcement Learning (IE 695)

Seminar: Computer Vision

Master and Bachelor Theses

If you are interested in writing a seminar, Bachelor or Master thesis with us, please feel free to contact us. The list below represents a highly incomplete set of the topics currently offered by our group, talk to us for more topics and additional information. Your own ideas and interests are welcome as well.

Using uncertainties to improve panoptic segmentation (contact: Shashank)
Optical Flow NeRFs (contact: Prof. Dr.-Ing Margret Keuper)
Compressing LLMs for Edge Devices (contact: Shashank)
Structured Pruning of Vision Models (contact: Shashank)
Zero Cost Proxies for evaluating Optical Flow Estimation methods (contact: Shashank)

Completed Student Projects

Team Project: Julian Yuya Caspary, Luca Schwarz and Xinyan Gao: FLOWBENCH: A ROBUSTNESS BENCHMARK FOR OPTICAL FLOW ESTIMATION. November, 2024

Publications

News:

S. Agnihotri, J. Grabinski, M. Keuper: Improving Feature Stability during Upsamping -- Spectral Artifacts and the Importance of Spatial Context, accepted at ECCV 2024.

K. Bäuerle, P. Müller, S. M. Kazim, I. Ihrke, M. Keuper: Learning the essential in less than 2k additional weights – a simple approach to improve image classification stability under corruptions, [openreview] [pdf] [bib], Transactions on Machine Learning Research (TMLR), 2024.

J. Grabinski, J. Keuper, M. Keuper: As large as it gets – Studying Infinitely Large Convolutions via Neural Implicit Frequency Filters, [openreview] [pdf] [bib] [code], Transactions on Machine Learning Research (TMLR), 2024.

S. Agnihotri, S. Jung, M. Keuper: CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks (https://openreview.net/forum?id=CXZqGJonmt), ICML 2024.

Y. Zhou, M. Fritz, M. Keuper: MultiMax: Sparse and Multi-Modal Attention Learning (https://openreview.net/forum?id=IC9UZ8lm25), ICML 2024.

J. P. Schneider, M. Fatima, J. Lukasik, A. Kolb, M. Keuper, M. Moeller: Implicit Representations for Constrained Image Segmentation (https://openreview.net/forum?id=IaV6AgrTUp), ICML 2024.

Y. Li, M. Keuper, D. Zhang, A. Khoreva: Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive (https://arxiv.org/abs/2401.08815), ICLR 2024.

J. Lukasik, P. Gavrikov, J. Keuper, M. Keuper: Improving Native CNN Robustness with Filter Frequency Regularization, TMLR 2023.

Y. Li, M. Keuper, D. Zhang, A. Khoreva, Divide & Bind Your Attention for Improved Generative Semantic Nursing, BMVC 2023 (oral).

T. Medi, J. Tayyub, M. Sarmad, F. Lindseth, M. Keuper, FullFormer: Generating Shapes inside Shapes, GCPR 2023.

K. Prasse, S. Jung, Y. Zhou, M. Keuper, Local Spherical Harmonics Improve Skeleton-based Hand Action Recognition, GCPR 2023.

J. Lukasik, J. Geiping, M. Möller, M. Keuper, Differentiable Architecture Search: a One-Shot Method?, AutoML Conference workshop, 2023.

P. Müller, A. Braun, M. Keuper, Classification robustness to common optical aberrations, ICCV2023 AROW Workshop.

S. Agnihotri, K.V. Gandikota, J. Grabinski, P. Chandramouli, M. Keuper, On the unreasonable vulnerability of transformers for image restoration and an easy fix, ICCV2023 AROW Workshop

An Evaluation of Zero-Cost Proxies -- from Neural Architecture Search to Model Robustness, J. Lukasik, M. Möller, M. Keuper, accepted at GCPR, 2023.

An extended Benchmark Study of Human-Like Behavior under Adversarial Training, P. Gavrikov, J. Keuper, M. Keuper, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023.

Improving Primary-Vertex Reconstruction with a Minimum Cost lifted Multicut Graph Partitioning Algorithm, V. Kostyukhin, M. Keuper, I. Ibragimov, N. Owtscharenko, M. Cristinziani, Journal of Instrumentation (JINST) , 2023.

Neural Architecture Design and Robustness: A Dataset, S. Jung, J. Lukasik, M. Keuper, ICLR 2023.

Intra-Source Style Augmentation for Improved Domain Generalization, Y. Li, D. Zhang, M. Keuper, A. Khoreva, WACV 2023.

Trading-off Image Quality for Robustness is not necessary with Regularized Deterministic Autoencoders, A. Saseendran, K. Skubsch, S. Falkner, M. Keuper, NeurIPS 2022.

Robust Models are Less Over-Confident, J. Grabinski, P. Gavrikov, J. Keuper, M. Keuper, NeurIPS 2022.

Learning Where to Look -- Generative NAS is Surprisingly Efficient, J. Lukasik, S. Jung, M. Keuper, ECCV 2022.

FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting, J. Grabinski, S. Jung, J. Keuper, M. Keuper, ECCV 2022.

Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks, S. Jung, M. Keuper, ECML-PKDD 2022.

Higher-Order Multicuts for Geometric Model Fitting and Motion Segmentation, E. Levinkov*, A. Kardoost*, B. Andres and M. Keuper (*equal contribution), in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2022.3148795.

NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search, J. Siems, L. Zimmer, A. Zela, J. Lukasik, M. Keuper and F. Hutter, ICLR, 2022.

Shape your Space: A Gaussian Mixture Regularization Approach to Deterministic Autoencoders , A. Saseendran, K. Skubch, S. Falkner, M. Keuper, NeurIPS 2021.

Spectral Distribution aware Image Generation, S. Jung, M. Keuper, AAAI 2021.

Estimating the Robustness of Classification Models by the Structure of the Learned Feature-Space, K. Ho, F.-J. Pfreundt, J Keuper, M Keuper, https://arxiv.org/abs/2106.12303

Beyond the Spectrum: Detecting Deepfakes by Image Re-Synthesis, Y He, N Yu, M Keuper, M Fritz, accepted at IJCAI 2021

Uncertainty in Minimum Cost Multicuts for Image and Motion Segmentation, A Kardoost, M Keuper, arXiv preprint arXiv:2105.07469, accepted at UAI 2021

Multi-Class Multi-Instance Count Conditioned Adversarial Image Generation, A Saseendran, K Skubch, M Keuper, arXiv preprint arXiv:2103.16795

SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain, P Harder, FJ Pfreundt, M Keuper, J Keuper, arXiv preprint arXiv:2103.03000, accepted at IJCNN 2021

Learning embeddings for image clustering: An empirical study of triplet loss approaches, K Ho, J Keuper, FJ Pfreundt, M Keuper, 2020 25th International Conference on Pattern Recognition (ICPR), 87–94

Object Segmentation Tracking from Generic Video Cues, A Kardoost, S Müller, J Weickert, M Keuper, 2020 25th International Conference on Pattern Recognition (ICPR), 623–630

Smooth variational graph embeddings for efficient neural architecture search, J Lukasik, D Friede, A Zela, F Hutter, M Keuper, arXiv preprint arXiv:2010.04683, accepted at IJCNN 2021

You can also find a full list of my publications on Google Scholar.

Articles
Conference Publications
Books

Schäfer, B., Keuper, M. and Stuckenschmidt, H. (2021). Arrow R-CNN for handwritten diagram recognition. International Journal on Document Analysis and Recognition : IJDAR, 24, 3–17.
Keuper, M., Tang, S., Andres, B., Brox, T. and Schiele, B. (2020). Motion segmentation & multiple object tracking by correlation co-clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 140–153.
Ilg, E., Saikia, T., Keuper, M. and Brox, T. (2018). Occlusions, motion and depth boundaries with a generic network for disparity, optical flow or scene flow estimation. Computing Research Repository (CoRR), 2018, 1–26.
He, Y., Keuper, M., Schiele, B. and Fritz, M. (2017). Learning dilation factors for semantic segmentation of street scenes. Computing Research Repository (CoRR), 2017, 1–11.
Keuper, M. (2017). Higher-order minimum cost lifted multicuts for motion segmentation. Computing Research Repository (CoRR), 2017, 1–11.
Wannenwetsch, A. S., Keuper, M. and Roth, S. (2017). ProbFlow: Joint optical flow and uncertainty estimation. Computing Research Repository (CoRR), 2017, 1–18.

Primpeli, A., Bizer, C. and Keuper, M. (2020). Unsupervised bootstrapping of active learning for entity resolution. In , The Semantic Web : 17th International Conference, ESWC 2020, Heraklion, Crete, Greece, May 31-June 4, 2020, Proceedings (S. 215–231). Lecture Notes in Computer Science, Springer: Berlin [u.a.].
Kardoost, A. and Keuper, M. (2019). Solving minimum cost lifted multicut problems by node agglomeration. In , Computer Vision – ACCV 2018 : 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part IV (S. 74–89). Lecture Notes in Computer Science, Springer: Berlin [u.a.].
Broscheit, S., Gemulla, R. and Keuper, M. (2018). Learning distributional token representations from visual features. In , ACL 2018, Representation Learning for NLP : Proceedings of the Third Workshop : July 20, 2018 Melbourne, Australia (S. 187–194). , Association for Computational Linguistics: Stroudsburg, PA.
Ilg, E., Saikia, T., Keuper, M. and Brox, T. (2018). Occlusions, motion and depth boundaries with a generic network for disparity, optical flow or scene flow estimation. In , Computer Vision – ECCV 2018 : 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part XII (S. 626–643). Lecture Notes in Computer Science, Springer: Berlin [u.a.].
He, Y., Chiu, W.-C., Keuper, M. and Fritz, M. (2017). STD2P: RGBD semantic segmentation using spatio-temporal data-driven pooling. In , 30th IEEE Conference on Computer Vision and Pattern Recognition : CVPR 2017 : 21–26 July 2016, Honolulu, Hawaii : proceedings (S. 7158-7167). , IEEE: Piscataway, NJ.
He, Y., Keuper, M., Schiele, B. and Fritz, M. (2017). Learning dilation factors for semantic segmentation of street scenes. In , Pattern Recognition : 39th German Conference, GCPR 2017, Basel, Switzerland, September 12–15, 2017, Proceedings (S. 41–51). Lecture Notes in Computer Science, Springer: Berlin [u.a.].
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A. and Brox, T. (2017). FlowNet 2.0: Evolution of optical flow estimation with deep networks. In , 30th IEEE Conference on Computer Vision and Pattern Recognition : CVPR 2017 : 21–26 July 2016, Honolulu, Hawaii : proceedings (S. 1647-1655). , IEEE: Piscataway, NJ.
Keuper, M. (2017). Higher-order minimum cost lifted multicuts for motion segmentation. In , 2017 IEEE International Conference on Computer Vision : ICCV 2017 : proceedings : 22 – 29 October 2017, Venice, Italy (S. 4252 -4260). , IEEE: Piscataway, NJ.
Wannenwetsch, A. S., Keuper, M. and Roth, S. (2017). ProbFlow: Joint optical flow and uncertainty estimation. In , 2017 IEEE International Conference on Computer Vision : ICCV 2017 : proceedings : 22 – 29 October 2017, Venice, Italy (S. 1182-1191). , IEEE: Piscataway, NJ.

Gemulla, R., Ponzetto, S. P., Bizer, C., Keuper, M. and Stuckenschmidt, H. (eds.) (2018). LWDA 2018 : Proceedings of the Conference “Lernen, Wissen, Daten, Analysen”, Mannheim, Germany, August 22–24, 2018. Aachen, Germany: RWTH Aachen.