Architectural Safety Mechanisms for Multi-Agent Clinical LLM Systems Under Knowledge Base Distribution Shift
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Objective
To evaluate whether multi-agent LLM architectures with explicit safety verification maintain guideline compliance when their clinical knowledge bases undergo temporal or institutional distribution shift.
Materials and Methods
We designed a controlled evaluation framework using 50,000 synthetic type 2 diabetes patients with CKD and hypertension comorbidities (500 per experimental condition). Four architecture modes (single-agent, naive RAG, linear multi-agent, stateful graph with safety floor) were tested under four shift regimes: baseline, temporal drift (updated eGFR thresholds), institutional vocabulary transformation (11 term-pair substitutions producing 0.36 cosine similarity degradation), and metadata erasure. The clinical task was medication reconciliation with contraindication detection. Two embedding models (all-MiniLM-L6-v2, PubMedBERT) and two LLM backends (Llama3-8B, Mistral-7B) were compared.
Results
Under institutional vocabulary shift, the linear pipeline’s Guideline Compliance Score dropped from 1.00 to 0.36 because retrieval degradation rendered critical contraindication guidelines unretrievable. The stateful graph architecture maintained GCS = 1.00 across all shift conditions through its regime-aware safety floor, which operates independently of retrieval quality. This pattern held across both LLM backends and both embedding models. The safety mechanism added 32.2s latency per patient under shift versus 12.5s for single-agent mode.
Discussion
Architectural choice (specifically whether audit findings are routed back to the summary agent) determines compliance under shift more than retrieval quality or model scale. The safety floor’s value is compliance maintenance, not semantic fidelity improvement.
Conclusion
Stateful multi-agent graphs with programmatic safety floors bound error propagation under clinical knowledge shift. The framework is reproducible on consumer hardware with no external API dependencies.
Lay Summary
When AI systems help doctors review medications, they rely on up-to-date medical guidelines stored in a database. If those guidelines change (because recommendations are updated or a hospital uses different terminology) the AI can silently give outdated advice. We tested whether connecting multiple AI agents in a loop, where one agent checks another’s work against safety rules, prevents this problem. It does: even when the database becomes unreliable, the safety-checking agent catches dangerous advice before it reaches the doctor. The trade-off is that the system takes about 20 extra seconds per patient.
