AI in Teaching – Here to Stay?

Every Tuesday at 1:45 p.m. during the 2023 spring/summer semester, Professor Wessler and 25 students met in seminar room 308, building B6 on the University of Mannheim campus for the seminar “Filter Bubbles and Echo Chambers: Myth and Reality.” The room has white walls, gray tables, and light brown chairs. On the ceiling is a projector, and behind the area where the lecturer stands is a screen. A perfectly ordinary classroom setup. Only one thing made this seminar a little different. Namely, the presence of a very special guest: ChatGPT — a chatbot that uses AI to exchange text-based messages with users.

In the seminar, Wessler and his students explored whether users of online platforms are fed information that confirms their existing political views. “That reinforces their beliefs and means they no longer see other points of view. However, research from the last three years shows that although filter bubbles and echo chambers, as these phenomena are known, do exist, they are less significant and powerful than is often assumed in public debates,” said Wessler. ChatGPT only became freely available in late 2022 and Wessler had not planned on using it in his seminar to test how familiar AI was with the latest research. “But then at the start of the year there was growing discussion about the chatbot in wider society and at the university. People were talking about regulations or bans. And since I’m interested in AI as a social and cultural technology that students will need in the future, I decided to incorporate ChatGPT.” The students liked the idea too.

Working with an AI that is fed on many millions of texts from the internet and other sources is a good fit for a seminar about media consumption. The start of the seminar was structured the same as any other, beginning with student presentations followed by a discussion. Students were then assigned a set text to read in preparation for the next meeting. The last 20 minutes of the seminars were reserved for ChatGPT. Wessler’s browser was projected onto the large screen, so that students could watch in real time as he opened ChatGPT (version GPT-3.5) and typed prompts into the text entry field: “Compile a summary of research on echo chambers.” “How strong is the tendency toward filter bubbles on TikTok?” “To what extent does a populist communication style help create online echo chambers?” And then things got really exciting when the AI replied. Wessler and his students could watch as it “typed” its answers, which appeared syllable by syllable in the chat window.

“We tried to develop a detailed understanding of AI and test it in myriad ways. For instance, by entering a more precise prompt if we weren’t happy with the initial results,” explained Wessler. “Since the students had learned a lot about filter bubbles and echo chambers during the seminar, they were familiar with the latest research and could assess the AI’s responses. They only rarely found them satisfactory.” One reason is the outdated research that ChatGPT has access to. Version GPT-3.5 was only trained up until fall 2021 and there have been many subsequent developments in the research that the AI was unaware of. Another reason is that the AI either does not give sources, or gives ones that are incorrect. It is important to understand how AI works: ChatGPT is a purely language-based model that calculates the probability that a certain word will come next. It does not check whether a text is factually accurate.

Based on his experiences in the seminar, Wessler has clear views on how to approach AI, which he has shared in meetings of the university’s Senate. “We talked about ChatGPT on two occasions in the spring and I argued that we should experiment with it in seminars and lectures so that we can learn about the technology together,” said Wessler. How to deal with ChatGPT and its possible impact on exams now that it is freely available had, of course, already been a big topic of discussion at the university. Mannheim’s Teaching and Learning Center (ZLL) published a handout in May that explained to teaching staff how they could use ChatGPT if the need arose. Wessler can put a tick next to one suggestion from the handout: using subject-matter knowledge in seminars to analyze the quality of texts produced by ChatGPT. Wessler’s students learned how to critically engage with AI output — not just in the seminar but also in their take-home assignments, in which both they and ChatGPT had to answer a research question on the seminar topic. The students then had to evaluate the AI’s answer. “They learned that AI cannot correctly answer research questions. Students always also need to do their own review of the literature. So I don’t see any danger of students, at least in my seminar, using ChatGPT to do their whole assignment for them,” said Wessler. However, he is in favor of students being required to declare if they used ChatGPT for their work.

In spring 2023, Wessler’s use of ChatGPT in his seminar made him something of a pioneer — at least if you ignore the School of Business Informatics and Mathematics, where AI is part of the daily bread and butter. But a glance at the course list for the 2023 fall/winter semester shows several more professors following in Wessler’s footsteps: for instance, political scientist Professor Thomas König with his seminar “Selected International Relations Topics with ChatGPT” and Professor Andreas Witt, head of the Leibniz Institute for the German Language (IDS)’s digital linguistics department, with his seminar “Using Digital Language Processing: From Spellcheckers to ChatGPT.” Wessler is not currently giving another seminar, as he is away from the university on a research fellowship, where he is working on the use of AI in public communications. “But I found the seminar very valuable for myself and the students, and was also pleased with the results of the assignments. And the students thanked me for taking the initiative,” he explained. He can envision doing more seminars in the future — perhaps with ChatGPT participating in the discussion in its own right, or with more in-depth engagement with other language models.

Text: Luisa Gebhardt/December 2023