Megabouts: a flexible pipeline for zebrafish locomotion analysis
Curation statements for this article:-
Curated by eLife
eLife Assessment
This study introduces Megabouts, a transformer-based classifier for larval zebrafish movement bouts. This useful tool is thoughtfully implemented and has clear potential to unify analyses across labs. However, the evidence supporting its robustness is incomplete. How the method generalizes across datasets, how sensitive it is to noise, and the specific sources of misclassification are unclear. The method would also be strengthened by providing options for users to fine-tune the clusters under different experimental conditions, which would further enhance reliability and flexibility.
This article has been Reviewed by the following groups
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
- Evaluated articles (eLife)
Abstract
Accurate quantification of animal behavior is crucial for advancing neuroscience and for defining reliable physiological markers. We introduce Megabouts (megabouts.ai), a software package standardizing zebrafish larvae locomotion analysis across experimental setups. Its flexibility, achieved with a Transformer neural network, allows the classification of actions regardless of tracking methods or frame rates. We demonstrate Megabouts’ ability to quantify sensorimotor transformations and enhance sensitivity to drug-induced phenotypes through high-throughput, high-resolution behavioral analysis.
Article activity feed
-
-
-
eLife Assessment
This study introduces Megabouts, a transformer-based classifier for larval zebrafish movement bouts. This useful tool is thoughtfully implemented and has clear potential to unify analyses across labs. However, the evidence supporting its robustness is incomplete. How the method generalizes across datasets, how sensitive it is to noise, and the specific sources of misclassification are unclear. The method would also be strengthened by providing options for users to fine-tune the clusters under different experimental conditions, which would further enhance reliability and flexibility.
-
Reviewer #1 (Public review):
Jouary et al. present Megabouts, a Transformer-based classifier and Python toolbox for automated categorization of zebrafish movement bouts into 13 bout types. This is potentially a very useful tool for the zebrafish community. It is broadly applicable to a wide variety of behavioral paradigms and could help to unify behavioral quantification across labs. The overall implementation is technically sound and thoughtfully engineered. The choice of standard Transformer architecture is well-justified (e.g., it can handle long-term tracking data and process missing data, integrates posture and trajectory information over time, and shows robustness to variable frame rates and partial occlusion). The data augmentation strategies (e.g., downsampling, tail masking, and temporal jitter) are well designed to enhance …
Reviewer #1 (Public review):
Jouary et al. present Megabouts, a Transformer-based classifier and Python toolbox for automated categorization of zebrafish movement bouts into 13 bout types. This is potentially a very useful tool for the zebrafish community. It is broadly applicable to a wide variety of behavioral paradigms and could help to unify behavioral quantification across labs. The overall implementation is technically sound and thoughtfully engineered. The choice of standard Transformer architecture is well-justified (e.g., it can handle long-term tracking data and process missing data, integrates posture and trajectory information over time, and shows robustness to variable frame rates and partial occlusion). The data augmentation strategies (e.g., downsampling, tail masking, and temporal jitter) are well designed to enhance cross-condition generalization. Thus, I very much support this work.
For the benefit of the end users of this tool, several clarifications and additional analyses would be helpful:
(1) What is the source and nature of the classification errors? The reported accuracy is <80% with trajectory data and still <90% with trajectory + tail data.
(1a) Is this due to model failure (is overfitting a concern? How unbiased were the test sets?), imperfections of the preprocessing step (how sensitive is this to noise in the input data?), or underlying ambiguity in the biological data (e.g., do some "errors" reflect intermediate patterns that don't map neatly onto the 13 discrete classes)?
(1b) A systematic error analysis would be helpful. Which classes are most often confused? Are errors systematic (e.g., slow swims vs. routine turns) or random?
(1c) Can confidence of classification be provided for each bout in the data? How would the authors recommend that the end user deal with misclassifications (e.g., by manual correction)?
Overall, the end user would benefit greatly from more information on potential failure modes and their root causes.(2) How well does the trained network generalize across labs and setups? To what extent have the authors tested this on datasets from other labs to determine how well the pretrained model transfers across datasets? Having tested the code provided by the authors on a short stretch of x-y zebrafish trajectory data obtained independently, the pipeline generates phantom movement annotations. The underlying cause is unclear.
(2a) One possibility is that preprocessing steps may be highly sensitive to slight noise in the x-y positional data, which leads to noise in the speed data. The neural net, in turn, classifies noise into movement annotations. It would be helpful if the authors could add Gaussian noise to the x-y trajectory data and then determine the extent to which the computational pipeline is robust to noise.
(2b) When testing the pipeline, some stationary periods are classified as movements. Which step of the pipeline gave rise to the issue is unclear. Thus, explicit cross-lab validation and robustness tests (e.g., adding Gaussian noise to trajectories) would strengthen the claims of this paper.
(2c) Lastly, given the potential issue of generalization across labs, it would be helpful to provide/outline the steps for users in different labs to retrain and fine-tune the model.
-
Reviewer #2 (Public review):
Summary:
Overall, the manuscript is well organized and clearly written. However, in this reviewer's opinion, the manuscript suffers from multiple major weaknesses.
Strengths:
The strengths of the paper are unclear; they have not been articulated well by the authors.
Weaknesses:
The pipeline is designed to analyze larval zebrafish behaviors, which by definition is considered a highly specialized, if not niche, application. Hence, the scope of this manuscript is extremely narrow, and consequently, the overall significance and the broader impact on the field of behavioral neuroscience are rather low. Broadening the scope would significantly improve the manuscript's impact. Second, it was noted that the authors neglect to present an unbiased discussion of how their pipeline compares to well-established and …
Reviewer #2 (Public review):
Summary:
Overall, the manuscript is well organized and clearly written. However, in this reviewer's opinion, the manuscript suffers from multiple major weaknesses.
Strengths:
The strengths of the paper are unclear; they have not been articulated well by the authors.
Weaknesses:
The pipeline is designed to analyze larval zebrafish behaviors, which by definition is considered a highly specialized, if not niche, application. Hence, the scope of this manuscript is extremely narrow, and consequently, the overall significance and the broader impact on the field of behavioral neuroscience are rather low. Broadening the scope would significantly improve the manuscript's impact. Second, it was noted that the authors neglect to present an unbiased discussion of how their pipeline compares to well-established and time-proven pipelines used to track larval zebrafish behaviors. This reviewer also failed to detect any new biological insights presented or improvements compared to existing methods, further questioning the overall significance and impact of this manuscript. Finally, the core claim of the manuscript lacks meaningful experimental data that would allow an unbiased and more definitive evaluation of the claims made regarding the Megabouts pipeline. The critical experiment to achieve this would be to run an identical set of behavioral assays (e.g., PPI, social behaviors) on different platforms (e.g., a commercial and a non-commercial one) and then determine if Megabouts correctly analyzes and integrates the results. While this might sound to the authors like an 'outside the scope' experiment, this reviewer would argue that it is the only meaningful experiment to validate the central claim put forward in this manuscript.
-
Reviewer #3 (Public review):
In this manuscript, the authors introduce Megabouts, a software package designed to standardize the analysis of larval zebrafish locomotion, through clustering the 2D posture time series into canonical behavioral categories. Beyond a first, straightforward segmentation that separates glides from powered movements, Megabouts uses a Transformer neural network to classify the powered movements (bouts). This Transformer network is trained with supervised examples. The authors apply their approach to improve the quantification of sensorimotor transformations and enhance the sensitivity of drug-induced phenotype screening. Megabouts also includes a separate pipeline that employs convolutional sparse coding to analyze the less predictable tail movements in head-restrained fish.
I presume that the software works as …
Reviewer #3 (Public review):
In this manuscript, the authors introduce Megabouts, a software package designed to standardize the analysis of larval zebrafish locomotion, through clustering the 2D posture time series into canonical behavioral categories. Beyond a first, straightforward segmentation that separates glides from powered movements, Megabouts uses a Transformer neural network to classify the powered movements (bouts). This Transformer network is trained with supervised examples. The authors apply their approach to improve the quantification of sensorimotor transformations and enhance the sensitivity of drug-induced phenotype screening. Megabouts also includes a separate pipeline that employs convolutional sparse coding to analyze the less predictable tail movements in head-restrained fish.
I presume that the software works as the authors intend, and I appreciate the focus on quantitative behavior. My primary concerns reflect an implicit oversimplification of animal behavior. Megabouts is ultimately a clustering technique, categorizing powered locomotion into distinct, labelled states which, while effective for analysis, may confuse the continuous and fluid nature of animal behavior. Certainly, Megabouts could potentially miss or misclassify complex, non-stereotypical movements that do not fit the defined categories. In fact, it appears that exactly this situation led the authors to design a new clustering for head-restrained fish. Can we anticipate even more designs for other behavioral conditions?
Ultimately, I am not yet convinced that Megabouts provides a justifiable picture of behavioral control. And if there was a continuous "control knob", which seems very likely, wouldn't that confuse the clustering process, as many distinct clusters would correspond to, say, different amplitudes of the same control knob?
There has been tremendous recent progress in the measurement and analysis of animal behavior, including both continuous and discrete perspectives. However, the supervised clustering approach described here feels like a throwback to an earlier era. Yes, it's more automatic and quantifiable, and the amount of data is fantastic. But ultimately, the method is conceptually bound to the human eye in conditions where we are already familiar.
-
