CASTLE: a training-free foundation-model pipeline for cross-species behavioral classification

Accurately quantifying animal behavior at scale and without manual labels is a long-standing bottleneck for neuroscience and ethology. We present CASTLE (Combined Approach for Segmentation and Tracking with Latent Extraction), an end-to-end training-free, label-free pipeline that combines state-of-the-art segmentation, object tracking, and foundation model-based feature extraction to generate detailed, orientation-invariant descriptors of movement. Hierarchical clustering of these descriptors through an interactive GUI yields distinct behavioral classes without predefined categories, revealing previously unrecognized behavioral motifs. CASTLE ’s versatility is demonstrated across model organisms, including skilled reaching and open-field exploration in mice, spontaneous locomotion and grooming in fruit flies, and foraging in C. elegans , consistently exceeding 90% agreement with expert human labels and even identifying disease-relevant phenotypes in a Parkinsonian mouse model. Notably, CASTLE ’s automatically tracked movements match conventional pose-tracking methods in neural decoding accuracy, confirming that its label-free approach does not compromise neurophysiological insights. A user-friendly graphical interface further makes this advanced analysis accessible to non-computational researchers, establishing CASTLE as a broadly applicable, scalable framework for cross-species behavioral phenotyping.