Can Machines Interpret My Smile Correctly?

Psychologists at the University of Mannheim have investigated the recognition of emotions by computer programs in a series of studies. Their conclusion: This emerging technology has considerable potential for use in psychological research and could possibly, for example, enable non-contact measurements in emotions research in the future. However, further improvements to the sensitivity of the programs are still needed.

What was science fiction only a short while ago is already mundane and ordinary: Smartphones recognize their owners’ faces. But will electronic gadgets soon be able to read our emotions, too? Mannheim psychologists Professor Georg W. Alpers, Dr. Tim Höfling, and their team have looked into this question in a series of studies. They have carried out a series of experiments in interdisciplinary cooperation projects with other universities to probe the accuracy of this innovative technology and explore its potential for various applications. The results of this work have now appeared in four international journals. 

The researchers were able to show that algorithms in special software can also recognize emotions portrayed in less standardized ways by actors almost as well as humans can. The software performed similarly well when asked to detect intense facial expressions portrayed by ordinary laypeople – not trained actors – in a typical lab setting. 

But artificial intelligence is still far from able to recognize every expression: The algorithms are not yet sensitive enough to detect weak emotional reactions – especially not when people are keeping their facial expressions under control or suppressing them. “While the sensitivity of the computer programs is still limited, they cannot yet completely replace more established research methods,” lead investigator Dr. Tim Höfling sums up. 

The studies show that the fast-evolving technology of facial expression recognition has considerable potential to prove useful in psychological research and its fields of application. Researchers could be able to read emotions straight from the faces of research subjects and patients in the future and to record them without time-consuming surveys or observations. This presents especially interesting opportunities for working with groups of sensitive patients that cannot easily be wired up with sensors. The technology is also suitable for online research, which proved to be of particular significance during the Covid pandemic. 

Text: Yvonne Kaul / October 2022