Short Report: Comparison of three methods for identifying implementation determinants to measurement-based care

Background Identifying implementation determinants, also referred to as barriers and facilitators, is considered a critical component of implementation science. There are many emerging methods for identifying implementation determinants, yet very few evaluations of how these methods complement or diverge from one another. The objective of this report is to compare three methods for identifying determinants in the context of the Novel Methods for Implementing Measurement-Based Care with Youth in Low-Resource Environments study. Methods Rapid evidence synthesis involves a targeted review of empirical literature. Rapid ethnographic assessment uses site visits, interviews, and observations to develop an insider’s perspective. Design probes engage participants in prompted activities (e.g., journaling, taking photos) to surface insights from their lived experience. We compared convergence of determinants identified by each method using a Jaccard plot and pairwise Jaccard indices. Results All three methods combined produced a list of 42 determinants. Rapid evidence synthesis surfaced 29 (69%) determinants, including 8 solely identified by this method. Rapid ethnographic assessment surfaced 35 (83%) determinants, with 4 solely identified by this method. Design probes surfaced 23 (66%) determinants and did not surface any unique determinants. A total of 14 (33%) determinants were identified by all methods. Pairwise Jaccard indices indicated the strongest convergence between rapid ethnographic assessment and design probes ( J  = .66) and rapid evidence synthesis and rapid ethnographic assessment ( J  = .52). Convergence between rapid evidence synthesis and design probes ( J  = .37) was more modest. Discussion This study describes the convergence of implementation determinants surfaced using three methods. We found substantial overlap between methods, with one third of determinants surfaced by all three methods. Despite this overlap, each method added unique insights. Rapid evidence synthesis surfaced determinants from the literature and theory that were less likely to be identified by rapid ethnographic assessment or design probes. Rapid ethnographic assessment had the highest yield of determinants. Design probes highlighted participant-driven perspectives that overlapped substantially with rapid ethnographic assessment; notably, design probes did not surface any new determinants. These results reinforce the complementary nature of multi-method determinant assessment while highlighting tradeoffs researchers must weigh when selecting determinant identification methods. Trial registration Clinicaltrials.gov. NCT05644756. Registered 11/18/2022. This trial was retrospectively registered, https//classic.clinicaltrials.gov/ct2/show/NCT05644756