Does Trial Selection Improve the Reliability and Validity of Attentional-Control Measures? A Simulation and Systematic Reanalysis of Existing Datasets [Stage 1 Registered Report]

Attentional control refers to the ability to maintain a goal and goal-relevant information in the face of distraction. Typically, measures of this ability compare performance on incongruent trials that induce response conflict with congruent trials that do not. Previous research has highlighted the difficulty of obtaining attentional-control measures that have good reliability and that correlate across different tasks. To address this difficulty, Moretti et al. (2025, Journal of Experimental Psychology: Learning, Memory, and Cognition) have selected and analyzed only those trials with the highest response conflict. That is, in two tasks, they selected and analyzed the trials following congruent trials and the trials with a fast response. They concluded that their approach resulted in increased reliability and correlation estimates. While promising, the results raise several questions. Why was the increase in reliability and correlation estimates observed not only for fast but also for slow trials? Do the results stem from the corrections applied to compensate for the trial reduction inherent in the selection? Do the results generalize to other tasks and designs? The purpose of the present study is to address these questions. To answer the first question, we conducted a simulation. For the second and third questions, we plan to compute reliability and correlations on several existing datasets using hierarchical models. These models have the advantage of accounting for the trial reduction without corrections. Overall, the present study should clarify whether selecting trials improves the measurement of attentional control.