What to check before using AI-based virtual patients in clinical psychology training: an ablation audit of four personas

Background. Virtual patients built on large language models are spreading through clinical psychology training, and anyone adopting one needs to know what to check. The usual question—whether the patient seems realistic—runs together two properties that can come apart: whether the persona is recognisable from one session to the next, which makes the stimulus standardised, and whether it uses the language typical of its disorder, which makes it representative of the condition taught. This study measures each separately. Methods. We analysed 106 transcripts of psychotherapy training sessions conducted by 59 postgraduate students with four virtual patients: María (depression), Natalia (generalised anxiety), Lian (posttraumatic stress disorder) and Adrián (an obsessive-compulsive profile). Three had been validated earlier by independent evaluators using diagnostic instruments. Our procedure removes whatever would account for a result without recourse to the virtual patient—trainee behaviour, the style prescribed in the prompt, prompt variant, the persona’s biography, and its most distinctive vocabulary—and checks whether the result survives. Results. All four personas proved recognisable, and recognition withstood every removal. A held-out session was assigned to its persona 93.4% of the time after deleting biography and the 300 most discriminative terms, and 82.8% of the time (28.7% majority-class baseline) using only patient language, without prescribed style and with prompt variant controlled. Trainee behaviour did not identify the persona (40.6%; confidence interval spanning the baseline), which places the signal in the virtual patient rather than the student. On disorder-typical language, three of the four showed the expected vocabulary, the strongest being Adrián, whose 66-word prompt lists no symptoms (Cliff’s δ = +1.00). Natalia was the exception: her prompt lists her symptoms and she had been diagnosed unanimously in the earlier study, yet none of her lexical markers reached significance. Conclusions. The four virtual patients work as recognisable, stable stimuli, and their identity rests neither on biography nor on trainee behaviour. Defining a persona did not require long prompts. Lexical analysis was informative for three of the four conditions but not for the one with the most diffuse clinical profile, so it should not serve as the only criterion.