Photograph of the Mannheim Palace, Ehrenhof

Exploring Psychometric Complexities in Intensive Longitudinal Data

Holger Brandt, 30 July 2025, 18:30

In this talk, I will address problems that arise when researchers collect longitudinal data and present the same items repeatedly. Questions that may arise are: How often should I present the questionnaire? Can I find out when participants become inattentive? What happens if an item changes its meaning for persons who responded to it very often? I will discuss these psychometric questions that accompany the collection of intensive longitudinal data, for example in ecological momentary assessments. In many situations, psychometric characteristics obtained from cross-sectional evaluations of questionnaires may be difficult to extrapolate to such longitudinal studies that use repeated measures in a high-frequency (such as once per day or more). I will provide different examples, including heterogeneous measurement invariance patterns that may vary across persons and/or time, the occurrence of careless and insufficient effort behavior (inattention), as well as structural changes in the dimensions that questionnaires are supposed to measure. Within the framework of Dynamic Structural Equation Modeling, I will provide ideas about how to test if such problems occur in the data set and how they can be incorporated to allow researchers to extract valid conclusions from the data. I will discuss challenges that these complex methods face such as necessary sample sizes, model fit, and optimal prior tuning in Bayesian implementations.