Online behavioral studies are vulnerable to agentic AI
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
We recently warned against the potential danger of LLMs and agentic AI for online behavioral research (Van der Stigchel et al., 2026). Using response time distributions, normal quantiles, and autocorrelation across trials, we suggested that such bots may already have entered Prolific in one of our datasets. Chetverikov (2026) convincingly demonstrated that these markers are insufficient in establishing the presence of bots in our data. Unfortunately, this does not mean that online behavioral studies are safe from LLM-built bots. In this letter, we demonstrate with videos that bots created through prompting alone can perform online behavioral experiments across three prototypical tasks. As such, our main warning (Van der Stigchel et al., 2026) remains.