Navigating analytical challenges in clinical trials using the multiverse approach

Making decisions regarding data processing and analysis are crucial steps toward extracting insights from data in clinical trials. Trial registries like clinicaltrials.gov promote transparency about these decisions and encourage making them in advance. However, clinical studies often face decisions with multiple reasonable options outside the bounds of preregistration, such as when studies conduct post hoc analyses, deviate from preregistered plans, or simply were not preregistered. Additionally, even a priori decisions often have multiple reasonable options from which to choose. Methods that maximize transparency and minimize bias in such situations are needed. This paper advocates for applying a “multiverse” approach to analyzing such data from clinical trials. The multiverse approach simultaneously selects and analyzes the various reasonable options for each decision and presents results across all analysis “universes.” We highlight common challenges and decisions when analyzing clinical trial data, review and expand upon the multiverse approach and show how it can address these challenges, and demonstrate the approach using data from a small randomized psychotherapy trial for posttraumatic stress disorder. In the example presented, results were fully consistent across the multiverse for one outcome (posttraumatic stress symptoms), partially consistent for another (relationship satisfaction), and mostly inconsistent for a third outcome (fear of intimacy). The multiverse approach is a flexible and transparent analysis option for clinical trials in the presence of uncertainty regarding data processing and analytic choices.