Hybrid Active Learning with Privacy-Preserving Synthetic Data for Medical Multimodal LLM Enhancement

We propose a novel framework for enhancing medical multimodal learning by integrating hybrid active learning (HAL) with privacy-preserving synthetic data generation, addressing critical challenges of data scarcity and patient privacy. The framework reformulates the Large Language Model (LLM) module as a dynamic multi-agent system, where modality-specific models collaborate through reinforcement learning to optimize both diagnostic accuracy and privacy compliance. The HAL component strategically selects the most informative unlabeled samples by combining uncertainty and diversity metrics, thereby minimizing annotation costs while maximizing model performance. Furthermore, synthetic medical data is generated under rigorous local differential privacy guarantees using a modified GAN architecture, ensuring that synthetic samples do not replicate real patient information. The multi-agent reinforcement learning mechanism dynamically adjusts key parameters, such as the trade-off between active learning criteria and privacy constraints, enabling adaptive optimization during fine-tuning. Experimental validation on multimodal medical datasets demonstrates significant improvements in diagnostic accuracy compared to conventional methods, particularly in low-data regimes. The proposed framework not only mitigates privacy risks inherent in medical data but also enhances the robustness of multimodal fusion by aligning cross-modal representations. This work represents a significant advancement in medical AI by unifying active learning, privacy preservation, and adaptive optimization into a single cohesive system, with broad applicability to clinical decision support and automated diagnostics.