Data Auditing and Quality Assurance in a Federated Learning Consortium; Getting the Best of Both Worlds from Cross-Institutional and In-House Data Quality Inspection
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Introduction
Rare and heterogeneous disease research increasingly relies on privacy-enhancing technologies such as federated learning (FL) to enable cross-institutional collaboration across fragmented datasets. However, data quality assurance in FL can be limited, as individual-level data may not be directly accessible. While schema-dependent inspection offers a partial solution, it requires standardised schemas or resource-intensive frameworks, hindering scalability and collaboration. To address these challenges, we implemented two complementary dashboards and evaluated their interplay.
Methodology
We adapted a recognised data quality control framework for re-use of electronic health record (EHR) data using breast cancer records from Centre Léon Bérard into two dashboards: (1) an in-house dashboard for intra-clinic quality assessment, and (2) a federated dashboard for inter-clinic quality assessment. Artificial inconsistencies were introduced into distributed datasets mirroring the inhouse source to evaluate detection capabilities.
Results
The in-house dashboard provided granularity and reliability, pinpointing individual-level inconsistencies, while the federated dashboard enabled cross-institutional pattern detection – trade-offs inherent to their designs. The federated system revealed ecosystem-wide trends inaccessible to single-institution tools, whereas the in-house dashboard provided local validation and thoroughness.
Discussion
Our findings confirm a complementary relationship: FL dashboards provide scalable, collaborative oversight but may require additional quality checks, while in-house tools ensure thoroughness at the cost of scalability. A combined model seemingly offers the optimal balance, accommodating both institution-specific needs and collaborative research requirements in evolving, multi-institutional ecosystems.
