Factors Influencing Accuracy, Interpretability and Reproducibility in the use of Machine Learning in Biology

The complexity and variability of biological data has promoted the increased use of machine learning methods to understand processes and predict outcomes. These same features complicate reliable, reproducible, interpretable, and responsible use of such methods, resulting in questionable relevance of the derived outcomes. Here we systematically explore challenges associated with applying machine learning to predict and understand biological processes using a well characterized in vitro experimental system. We evaluated factors that vary while applying machine learning classifers: 1) type of biochemical signature (transcripts vs. proteins), data curation methods (pre- and post-processing), and 3) choice of machine learning classifier. Using accuracy, generalizability, interpretability, and reproducibility as metrics, we found that the above factors significantly modulate outcomes even within a simple model system. Our results caution against the unregulated use of machine learning methods in the biological sciences, and strongly advocate the need for data standards and validation tool-kits for such studies.