Online behavioral studies are safe for now: Unusual RTs do not imply bots (A reply to Van der Stigchel et al., 2026)
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Van der Stigchel et al. (2026) recently raised concerns about bots in online behavioral data collection. This is a high-stakes claim that has already garnered considerable attention on social media and could influence editorial and reviewing practices. Reviewers might, for example, cite this letter to request that authors verify their online data for bot activity. Therefore, it is crucial that the concerns raised in the letter are well-supported. In this Reply, I reanalyze their data and compare it to an offline dataset from a similar task to show that these concerns are unfounded. I also outline general reasons to be wary of suspecting bots based solely on behavioral metrics, and argue that developing a bot for a given task would be economically unfeasible.