Scalable Drift Monitoring in Medical Imaging AI

The integration of artificial intelligence (AI) into clinical medical imaging has advanced diagnostics but also poses challenges inmaintaining long-term reliability as input data change. To address this, we present MMC+, a modular framework for scalabledrift monitoring that builds on our CheXstray framework for real-time drift detection using multi-modal data concordance (MMC). MMC+ provides a flexible architecture that enables cost-effective continuous performance monitoring in real-world healthcare settings. The framework’s capabilities include aggregating information from multiple data streams from statistical tests ordistance metrics, leveraging foundation model embeddings for generalization across sites without requiring site-specific training, and quantifying uncertainty to better communicate drift significance. Validated with data from Massachusetts General Hospitalduring the COVID-19 pandemic, MMC+ effectively detects significant data shifts that correlate with performance changes and indicates when AI systems may deviate from acceptable performance bounds. The code for MMC+ is publicly available (Link: https://github.com/FJDorfner/MedImaging-ModelDriftMonitoring/tree/MMC_plus), promoting broader adoption of AI solutions in clinical settings. Our framework significantly advances drift detection while promoting transparency and collaboration in clinical AI research.