Visual Human Awareness Estimation for Mobile Robots
Background
As mobile robots increasingly operate in environments shared with humans, they need to do more than simply detect people and avoid collisions. Successful human-robot interaction also requires a robot to understand how people perceive and respond to its presence.
Consider a mobile service robot approaching a pedestrian in a hallway. The pedestrian may already have noticed the robot and intentionally make space, or they may be distracted, looking in another direction, or unaware that the robot is approaching.
The European Master Team Project brings together students from the University of Mannheim and Babeş-Bolyai University in Cluj-Napoca, Romania, to investigate this problem in an international and interdisciplinary project environment. The project is supervised by the Institute for Enterprise Systems (InES)and individual project coordinators and is conducted in cooperation with the Robotics Lab of the CORE research group at Clausthal University of Technology (https://www.core-network.ai/core-labs).
Students will collaborate across universities throughout the semester, combining research-oriented work with practical software development and experimentation in robotics and computer vision.
Project Description
The goal of the project is to develop a camera-based perception system that estimates whether people near a mobile robot are aware of the robot and how likely they are to react to it.
The system should analyze visual information obtained from cameras mounted on or associated with a mobile robot. Rather than relying on a single indicator, students will investigate how several observable human cues can be combined to estimate awareness. The resulting perception system should transform these observations into an interpretable estimate of human awareness.
A major aspect of the project will be investigating how individual perception components can be combined into a robust estimation pipeline. Students may explore established computer-vision techniques, pretrained models, multimodal approaches, or machine learning methods and evaluate their suitability for real-world robotic applications.
During the project, the student team is expected to work on several stages of the development process. In addition to developing an experimental method, the project result should be an optimized framework, e.g., a Python library, that can be adopted by practitioners in real world applications. This practical applicability is motivated by an additional visit to the robotics lab in Goslar at the end of the semester, where students will be able to see their developed solution in action.
Requirements:
Python programming skills and familiarity with libraries such as PyTorch
Familiarity with Git and GitHub
Knowledge of machine learning techniques (e.g., IE500 Data Mining, IE675b Machine Learning)
Knowledge of deep learning, particularly in the field of computer vision, is desirable (e.g., IE678 Deep Learning, CS646 Higher Level Computer Vision, CS668 Generative Computer Vision Models)
Ability to work independently as well as in a team, strong analytical thinking skills
What this project offers you
Working on a highly relevant research problem with other motivated students
A project motivated by a real use case instead of theory alone
Building skills in research and development which are relevant for your future
Funded trips to the partner university in Cluj and the robotics lab in Goslar
We look forward to receiving applications from talented and motivated students who are eager to sharpen their data science expertise and learn how to write code that lasts beyond the scope of a single project.