Exploring the Impact of Design Choices on Careless Responding with Confirmatory Mixture Models

Careless and insufficient effort responding (C/IER) on self-report measures results in responses that do not accurately reflect the constructs being measured, posing a major threat to the quality of survey data. Confirmatory mixture models, which translate theoretical considerations on respondent behavior into constrained component models, offer sophisticated tools for studying the occurrence of C/IER, along with its determinants and contextual correlates. Using two case studies based on experimental survey data, I (a) illustrate how confirmatory mixture models can be formulated for different data sources and collection methods and (b) demonstrate how they can inform guidelines for study designs that mitigate C/IER and yield higher-quality data. The first study, focusing on cross-sectional data, examines whether increasingly used visual analogue scales elicit more or less C/IER than traditional Likert-type scales, utilizing models that account for the unique properties of each scale type. The second study uses screen time data from an experimental ecological momentary assessment to investigate how questionnaire length and sampling frequency impact C/IER occurrence in this high-frequency longitudinal data collection method. I conclude with a cautionary note on the risks of erroneous conclusions when model assumptions are violated.