AI-Powered Ecosystem for Multilingual Diagnostics and Adaptive Specialty Mapping

In this paper, we propose an integrated AI-driven framework to address critical challenges in healthcare diagnostics and patient management by combining advanced natural language processing, speech recognition, and image analysis techniques. Our framework leverages Google Cloud Vision for accurate text extraction from medical documents, Gemini AI for generating multilingual patient summaries, and OpenAI’s Whisper for real-time audio transcription to enable seamless symptom reporting. We introduce a state-machine-based conversational system, enhanced by regex-based transcription normalization and reinforcement learning, to guide patients through symptom analysis and doctor assignment. For symptom-to-specialty mapping, we establish a reinforcement learning foundation that utilizes Firebase-stored data, periodically updated via Gemini AI, to reduce dependency on real-time external API calls while maintaining mapping accuracy. Additionally, a Flask-based web application provides an intuitive interface for patients to upload medical records and receive personalized summaries in regional languages, such as Kannada and Tamil. The system ensures scalability and security through Firebase Firestore and Google Cloud Storage integration, alongside robust user authentication. This framework offers a comprehensive toolkit for improving healthcare accessibility and operational efficiency, particularly in multilingual and resource-constrained environments. Experimental results demonstrate enhanced accuracy in symptom mapping and streamlined patient onboarding, paving the way for scalable, patient-centric healthcare solutions.