Development of Group Brief Cognitive Behavioral Therapy Fidelity Assessment Instruments

Purpose: Treatment fidelity is important for both research and clinical practice, including adapting new approaches for group delivery of Brief Cognitive Behavioral Therapy (G-BCBT) for suicide prevention. This study describes the development and preliminary field test of 12 fidelity assessment instruments (FAIs) for G-BCBT, for use in a non-inferiority trial comparing G-BCBT to Dialectical Behavioral Therapy (DBT) skills group. Method: This study included 141 active-duty service members (69 male, 72 female). Participants were randomized to one of two conditions with a total of 10 G-BCBT and 10 DBT group therapy cohorts. Twelve G-BCBT FAIs were developed (one per session) and used by two expert raters to evaluate G-BCBT fidelity. Results: Range of percent agreement was high among expert raters, with 86.3% of agreement scores exceeding 85%. Results for inter-rater reliability were mixed with several scores reflecting perfect agreement and others achieving fair to moderate levels of agreement. However, most inter-rater reliability metrics demonstrated less than chance levels of agreement, suggesting possible halo or ceiling effects. Conclusion: Initial field testing of G-BCBT FAIs provided mixed support for use. Recommendations to improve FAI use and inter-rater reliability metrics are discussed as well as directions for future research.