Evaluation of Elicit as an adjunct search, screening, and data extraction tool for systematic reviews
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
Systematic reviews are critical for informing evidence-based medicine, policy, and future research. However, conducting a systematic review is labour- and cost-intensive and error-prone. Elicit is an AI tool that claims to replicate the systematic review process.
Methods
Our study investigated Elicit’s ability to replicate a systematic review by comparing its results with those of an existing systematic review recently published by our research group. We calculated accuracy, precision, sensitivity, error rate, and person-time savings at each stage of the systematic review.
Results
At search, Elicit had poor overlap, finding 158 of the 5285 studies. For abstract and full-text screening, Elicit had moderate sensitivity and accuracy but poor precision, with a skew toward false positives. Elicit’s data extraction tool was promising, with sentence-specific citations, and provided helpful information for 85% of values even when it did not find exact matches. Elicit offered person-time savings, taking 3.16 hours to complete the full systematic review process.
Conclusions
Elicit shows promise as an adjunct screening and extraction AI tool, but in its current form, it is not ready as a standalone tool to replace the systematic review process.
