Scoping review of software implementations of risk-of-bias tools and quantitative bias analysis methods for sample selection bias

Background Failure to appropriately account for selection bias may lead to erroneous conclusions. Risk-of-bias tools are recommended (in systematic reviews of medical studies) to identify potential selection bias, and quantitative bias analyses (QBA) are recommended to quantitatively assess the sensitivity of a (meta-analysis or individual) study’s conclusions to plausible assumptions about the selection mechanism. While risk-of-bias tools are widely available, their effective use is complicated by the large number of available instruments with varying granularity and a high reliance on investigator expertise. QBAs are not routinely implemented, due to a lack of knowledge about accessible methods and software. Methods We conducted a scoping review of software implementations of risk-of-bias tools and QBA methods, for sample selection bias, published from January 2004 through August 2025. Inclusion criteria were validated risk-of-bias tools and software not requiring adaptation (i.e., code changes) before application, still available in 2025, and accompanied by documentation. Key properties of each risk-of-bias and software tool were identified. Results We identified 24 risk-of-bias tools. All consisted of one to five signalling questions relating to selection bias and covered a wide range of study designs (e.g., randomised controlled trials, non-randomised and diagnostic accuracy studies). Question granularity varied and fewer than half of tools included a dedicated selection bias domain, with only three providing structured guidance for synthesizing the evidence from these questions into a final risk judgment on selection bias. For QBA, we identified nine software programs (one web-based, five R packages and three Stata commands). Six programs were only applicable when estimating the effect of a binary exposure on a binary outcome. Three programs implemented a probabilistic QBA, one a multidimensional QBA, and one a tipping point QBA. The remaining programs required the user to supply their own code to fully implement a QBA. Conclusions While many risk-of-bias tools are available, the degree of guidance provided varies drastically, and the overall assessment of selection bias depends heavily on the investigator's interpretation. Greater provision of QBA software to selection bias, along with detailed QBA guidelines, would facilitate the wider uptake of QBA among future studies.