Can Machines Read My Face?

Press release, 12 April 2022
Print Version (PDF)

What was recently considered science-fiction, is now common: The smartphone recognizes the face of its owner. But will electronic devices soon be able to read our emotions? In numerous studies, psychologists of the University of Mannheim, led by Professor Dr. Georg W. Alpers und Dr. Tim Höfling, tried to answer this question. In interdisciplinary collaborations with other higher education institutions, the researchers conducted a series of experiments and published the findings in four international journals. In these studies, they tested the accuracy of this innovative technology and analyzed its potential for various applications. The technology is able to recognize intensive and, in particular, standardized facial expressions very well. However, it remains unclear if the technology is also able to recognize more naturalistic emotional facial expressions.

The researchers could demonstrate that the algorithms of specialized software were also able to recognize less standardized emotional expressions of actors. The performance of the algorithms was almost as good as the performance of humans (1). The result was similar for intensive facial expressions made by normal people, not actors, in a typical lab setting (2). Even spontaneous emotional reactions, for example, reacting to a pleasing picture, can be measured and analyzed (3).

However, the machine is not able to detect everything: Algorithms are not sensitive enough to recognize subtle emotional reactions, especially when people control or suppress their facial expressions. The software is not able to detect such emotional reactions. Such reactions can only be measured in a biopsychological laboratory by directly measuring the facial muscle activity (4).

The studies show that the emerging technology in the field of recognizing facial expressions holds great potential for psychological research and its areas of applicability. Researchers could be able to read emotional facial expressions of their test subjects and patients without extensive surveys or observations. This is particularly relevant for sensitive patients for whom cables may be difficult. The technology is also suitable for online research, which is particularly relevant during the pandemic.

However, the studies also show the limitations of the technology. “As long as the sensitivity of the computer programs is restricted, established research methods cannot be fully replaced”, says Höfling, head of the study. The researchers also point out that ethical aspects are also relevant, since algorithms may also be misused for economic or political purposes.

More information:

1. Küntzler, T., Höfling, T. T. A., & Alpers, G. W. (2021). Automatic facial expression recognition in standardized and non-standardized emotional expressions. Frontiers in Psychology, 12, 1086. https://doi.org/10.3389/fpsyg.2021.627561
2. Höfling TTA, Alpers GW, Büdenbender B, Föhl U, Gerdes ABM (2022) What’s in a face: Automatic facial coding of untrained study participants compared to standardized inventories. PLOS ONE, 17(3): e0263863. https://doi.org/10.1371/journal.pone.0263863
3. Höfling, T. T. A., Gerdes, A., Föhl, U., & Alpers, G. W. (2020). Read my face: automatic facial coding versus psychophysiological indicators of emotional valence and arousal. Frontiers in Psychology, 11, 1388. https://doi.org/10.3389/fpsyg.2020.01388
4. Höfling, T. T. A., Alpers, G. W., Gerdes, A. B., & Föhl, U. (2021). Automatic facial coding versus electromyography of mimicked, passive, and inhibited facial response to emotional faces. Cognition and Emotion, 35(5), 874–889. https://doi.org/10.1080/02699931.2021.1902786

Contact:
Dr. Tim Höfling
Chair of Clinical, Biological Psychology and Psychotherapy
University of Mannheim
E-mail: thoeflinmail-mail.uni-mannheim.de

Professor Dr. Georg W. Alpers
Chair of Clinical, Biological Psychology and Psychotherapy
University of Mannheim
E-mail: alpersmail-uni-mannheim.de

Yvonne Kaul
Research communication
University of Mannheim
E-mail: kaulmail-uni-mannheim.de