On the Comparability of Studies in Predictive Ecotoxicology

We address several comparability issues in chemical hazard assessment through in silico models. We review the recent literature and show that studies predicting fish acute toxicity are not comparable based on the following criteria: (i) The datasets are different, representing different chemical and/or taxonomic spaces; (ii) Even if the dataset were the same, cleaning might be different; (iii) Even if the cleaning is the same, the data is split in a different manner. Additionally, the test set is often also used as a validation set. (iv) Even when the dataset and the splittings are the same, the metrics are not clearly defined or deceptively different (e.g., different definitions of R2 give different results). As a short-term solution and as the only course of action ensuring the effective progression of the field as a whole, we call for the usage of benchmark datasets, which provide well-defined data cleaning, versioning and splittings. This should be complemented by a clear definition of the used metrics, and a transparent reporting, which includes code and the use of reporting/data sheets.