Doc Bot: The Medical LLM Fine-tuned on LLaMA 3 8B Using LoRA and Insights from the Medical Field

Purpose General-purpose large language models (LLMs) often lack the specialized accuracy required for the medical domain. This research aims to address this gap by developing and evaluating DocBot, a medical LLM, to demonstrate that domain-specific fine-tuning can significantly enhance performance even with constrained computational resources. Methods We fine-tuned the Meta LLaMa 3.1 8B model using Low-Rank Adaptation (LoRA), a parameter-efficient technique. The model was trained on a curated dataset of 2,000 patient-doctor dialogues sourced from ClinicalTrials, EMEA, and PubMed, using a single Tesla T4 GPU. Performance was evaluated against the base LLaMa model using BERTScore, BLEU, and ROUGE metrics, with responses from verified medical professionals serving as the reference. Results DocBot demonstrated significant improvements over the base LLaMa 3.1 8B model across all evaluation metrics. Specifically, DocBot achieved a higher BERTScore F1-score (83.56% vs. 81.47%), indicating enhanced semantic accuracy, fluency, and alignment with expert-generated text. The gains in precision and recall further confirm the model's superior ability to generate relevant and comprehensive medical information. Conclusion The successful development of DocBot showcases the feasibility and impact of creating domain-optimized LLMs efficiently. The results highlight the potential for specialized models to serve as reliable tools for augmenting clinical decision-making and delivering accessible medical support, particularly in resource-limited environments, paving the way for further innovation in specialized AI applications.