Must Humans Have the Last Word?
People asked about AI-based decision-making processes do not see the use of algorithms as a critical problem in itself. Instead, they find it problematic that the results are not subject to enough human scrutiny. This is what a new study led by data scientists Professor Florian Keusch and Professor Frauke Kreuter has determined.
Credit: pw studio
The deployment of an algorithm by the Austrian labor market agency AMS in late 2020 to match the job and training opportunities shown to job seekers with their individual profiles sparked intense controversy. Many people criticized the approach taken because it drew on historical data and thus potentially put people who had already experienced labor market discrimination in the past at a disadvantage. Women, for example, automatically had points deducted from their profiles, and mothers were saddled with an extra deduction.
Algorithms are widely used – and controversially discussed – not only in the labor market, but also in banking, human resources, and medicine. The data scientist Professor Florian Keusch from the University of Mannheim and Professor Frauke Kreuter from LMU Munich have looked into how acceptable the public finds decisions made based on algorithms.
“The results of our study indicate that the use of algorithms without additional scrutiny of the process by humans is seen as especially problematic,” Keusch notes. “It is not the deployment of algorithms as such that is contentious,” the Mannheim professor adds.
The researchers conducted their study as an online survey with more than 4,000 participants in the context of the German Internet Panel (GIP). Participants had to answer questions about the fairness and acceptability of AI-assisted decisions in four different scenarios: approval of a financial product, job applications, prison sentences, and measures aimed at job seekers.
The use of AI is already a reality – at least to some extent – in all four areas today. Companies and state agencies make use of what is generally described as “automated decision-making” (ADM) primarily to increase the efficiency of decision-making processes and reduce the influence of decision-makers’ personal attitudes. It is still uncommon for a computer to make a decision unaided. But according to the study authors, it is certainly possible that certain processes will be entirely automated in the future.
Text: Yvonne Kaul / October 2022