Single-Item Reliability for Intensive Longitudinal Data

With the advancement of technology, measures are more and more often collected in real-time, resulting in intensive longitudinal data (ILD). ILD allow us to study dynamic properties of one or more variables, such as moment-to-moment fluctuations in affect or changes in heartbeat rates. Despite the widespread use of ILD, the issues of measurement error and reliability are often inadequately addressed, particularly in the presence of serial correlation and complex multilevel structures. This paper introduces a three-level linear mixed-effects model that explicitly accounts for the nested data structure and the serial correlation (i.e., moments within days, days within persons). The model incorporates two distinct autoregressive processes to capture serial correlations at both the moment and day levels, while simultaneously partitioning measurement error from within-person variability. Building on this model, we delineate two approaches to defining reliability: a variance-partitioning approach and a correlation approach. This framework yields a multifaceted set of reliability coefficients, enabling researchers to distinguish between between-person from within-person reliability, and to assess measurement consistency across different time scales (e.g., moment-to-moment and day-to-day). We further discuss the relationship between the proposed reliability measures and those derived from simpler models. We apply our model to study the reliability of a single negative affect item in an ILD study in which students were measured multiple times a day for several days before and after receiving their exam results.