Discrepancies in reporting of study design and analysis methods between protocols and reports of interrupted time series (ITS) studies: a methodological study

Background : Unexplained changes from a study’s analysis plan may increase the risk of bias in study results and undermine confidence in the findings. Discrepancies in methods between protocols and reports have not been examined in interrupted time series (ITS) studies. Methods : We searched for peer-reviewed protocols of ITS studies in 22 databases, and results reports addressing each protocol’s primary research question. We compared 20 design and analysis items between protocols and their reports, classifying an item as ‘discrepant’ when the report provided different details about the item compared to the protocol. We judged a discrepancy as ‘potentially important’ if it could have potentially impacted the results or the study conclusion. We recorded if authors provided justifications for discrepancies. Results : From 4,590 abstracts, after excluding ongoing studies, we identified 120 eligible ITS protocols, from which 44 protocols (37%) had at least one corresponding results report. Information about handling the complexities of time-series data were frequently missing from the protocol or report, or both, for example, methods of adjusting for autocorrelation (77% of studies, 34/44) and seasonality (82%, 36/44). Potentially important discrepancies were common for eligibility criteria (43%, 19/44), overall length of time series (39%, 17/44), length of each time segment (45%, 20/44), effect measures (25%, 11/44) and ITS analysis methods (23%, 10/44). Among studies with important discrepancies, justifications were missing in 50%-100% of cases. Conclusion : Study designs and analysis methods were often not, or insufficiently, reported in ITS studies. Changes to original analysis plans were also prevalent and often unjustified, precluding readers from judging the legitimacy of the changes. Protocols for ITS studies should provide detailed information about the design and analysis methods. Deviations from planned methods should be transparently reported with clear justifications.