Multi-Model LLM Architectures for Personalized Summarization and Relevance Ranking in Biomedical Literature
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
To develop and evaluate a personalized literature review system that efficiently processes and summarizes biomedical literature to provide timely, relevant insights for researchers.
Methods
The system integrates ontology-aware keyword extraction (MeSH/ACM constrained TF-IDF from CV/Research Statement), citation-informed retrieval (PubMed and NIH iCite API), and dual-model large language model (LLM) summarization (Google Gemini 2.0 flash, OpenAI GPT-4o-mini). These LLMs leverage advanced Transformer architectures, building on foundations such as BERT, BART, and BioBERT. A two-stage ranking algorithm combines Relative Citation Ratio (RCR) with cosine similarity.
Summary quality was evaluated using ROGUE-1/2/L and BERTScore. The system is deployed as a Streamlit web application.
Results
Across 20 biomedical queries, the system demonstrated strong average performance (BERT-F1≈ 0.86), with cosine similarity strongly correlating with summary quality. Human evaluation involving 10 users yielded average scores above 4.5/5 across summary fidelity and keyword relevance.
Conclusion
Hybrid ranking and ensemble LLM summarization significantly accelerate scientific sense-making. These findings suggest broad applicability to various domains beyond biomedicine.
