Large Language Models in Portuguese for Healthcare: A Systematic Review

Purpose: This study addresses Large Language Models (LLMs) pre-trained in Portuguese for healthcare applications, focusing on contextual embeddings. Research on LLMs for natural language processing (NLP) tasks in Portuguese is limited, especially within healthcare. However, LLMs demonstrate potential in clinical decision support, diagnosis assistance, patient care, and other healthcare applications. In view thereof, the present work assesses the current state of LLMs in Portuguese for healthcare. Methods: Our Systematic Literature Review (SLR) followed standard protocols: search, screening based on inclusion/exclusion criteria, quality assessment, data extraction, and analysis. Results: We identified 28 models, mostly based on BERTimbau, mBERT, and BioBERTpt. Adaptation strategies such as fine-tuning, domain-adaptive pre-training, training from scratch, and zero-shot learning have been the most prevalent. Several datasets have been used, including clinical records, social media, and scientific repositories. LLMs in Portuguese are being applied in mental health, general medicine, COVID-19, oncology, and other related areas, accomplishing classification tasks, followed by named entity recognition (NER), topic modeling, question answering, text generation, and conversational agents. Conclusion: Our study identified gaps and opportunities: (1) base models such as LLAMA, T5, ELECTRA, BART, XLM-R, Falcon, Mistral, BLOOM are unexplored yet; (2) there is a lack of detailed fine-tuning specifications, hindering reproducibility; (3) many healthcare fields are not even tackled; (4) clinical and hospital data have been widely used but not shared; (5) social media data need caution because it can introduce inconsistencies; (6) data privacy, especially de-identification and anonymization, have been largely overlooked; and (7) Brazilian healthcare data present large opportunities.