Drug Safety Warnings in English, Russian, Arabic and Hebrew Wikipedia: A Cross-Sectional Audit of 33 Drugs Across Three Risk Domains

Background

Wikipedia is a leading consumer health resource, yet its medical content is audited almost only in English, and drug-safety warnings may be weaker in other editions.

Objective

To test whether Russian, Arabic, and Hebrew Wikipedia articles carry less actionable drug-safety warnings than English articles.

Methods

We audited 33 drugs in four language editions: 24 risk-positive drugs (eight each for glucose-6-phosphate dehydrogenase [G6PD] deficiency, teratogenicity, and QT-interval prolongation, benchmarked to reference standards) and nine negative controls. Three large language models, used as a measurement instrument, coded each warning 0 (not mentioned), 1 (mentioned), or 2 (explicit directive). A pharmacist resolved disagreements for G6PD and teratogenicity; the 12 QT disagreements kept the majority code, with an extreme-code sensitivity analysis. The prespecified analysis was a hierarchical Bayesian cumulative-logit model with English as reference.

Results

Pooled cumulative odds ratios versus English were 0.41 (Russian), 0.25 (Arabic), and 0.31 (Hebrew), with posterior probabilities of 78%, 87%, and 87% that English warns more strongly; the 95% credible intervals included 1. Eight of nine language-by-domain estimates were below 1 (exception: Russian QT). Exploratory analyses suggested that the Russian gap mainly reflects missing or short articles, whereas Arabic articles less often escalated a mentioned risk to a directive. Wikidata’s contraindication property appears in 12 statements in total, none for the audited drugs.

Conclusions

Warnings were weaker in Russian, Arabic, and Hebrew than in English in eight of nine comparisons, with wide uncertainty from a small purposive panel and model codes not yet validated against human coding.

Key Points

• For 24 drugs with a known risk of one of three unrelated kinds of harm, Russian, Arabic and Hebrew Wikipedia articles were less likely than English articles to give readers a clear safety instruction in eight of nine comparisons, although the estimates were imprecise.

• Exploratory analyses suggest that in Russian the gap mostly reflects missing or short articles, whereas Arabic articles that mention a risk less often turn it into an instruction to avoid the drug.

• Wikidata’s dedicated contraindication field is almost unused; filling it for drugs with a known risk is a low-cost place to start closing the gap.

Plain Language Summary

Many people look up medicines on Wikipedia, and more and more people read its content indirectly, through AI chatbots and search summaries built on it. We checked whether Wikipedia warns readers about serious medicine risks equally well in English, Russian, Arabic, and Hebrew. We studied 33 medicines and three kinds of harm: a severe blood reaction in people with an inherited condition called G6PD deficiency, harm to an unborn baby, and a dangerous heart rhythm problem. For each article, we recorded whether it said nothing about the risk, mentioned the risk, or clearly told readers to avoid the medicine. Articles in Russian, Arabic, and Hebrew were less likely than English articles to give a clear warning, and this pattern appeared in eight of the nine combinations of language and kind of harm. Part of the gap came from articles that were missing or short. In Arabic, however, articles that mentioned a risk were still less likely to tell readers to avoid the medicine. Wikidata, the structured database that supports Wikipedia, has a dedicated field for recording the condition in which a medicine must not be used, but the field is used in only 12 places across the whole database and none of the 33 medicines has an entry, which makes it a simple place to start closing the gap.