Enhancement of inter-/intra-reader agreement using the Prostate Imaging Reporting and Data System version 2.1 for prostate cancer detection in magnetic resonance imaging/transrectal ultrasound software fusion prostate biopsy

Purpose: This study evaluated inter-/intra-reader agreement with PI-RADS version 2.1 to improve the detection rate of prostate cancer. Methods: We enrolled 210 patients who underwent multiparametric magnetic resonance imaging (mpMRI) for clinically suspected or diagnosed prostate cancer. Four readers, including two urologists, viewed patients’ mpMRI and scored PI-RADS between two sessions. Inter-and intra-reader agreements were evaluated using Fleiss's Kappa coefficient (κ), agreement coefficient 1 (AC1), and percentage of agreement(PA). Results: The overall inter-reader agreement between all readers was moderate (κ=0.466, AC1=0.522, and PA=0.610). The overall inter-reader agreement improved in the second session. The agreement for peripheral zone (PZ) lesions was higher than that for transitional zone (TZ) lesions. At a PI-RADS cut-off of 4, the agreement for PZ lesions was almost perfect (PA=0.888) and higher than that for TZ lesions. The inter-reader agreement for lesions with a PI-RADS ≥ 4 and Gleason score ≥ 7 was almost perfect (AC1=0.960 and PA=0.964). The intra-reader agreement for lesions overall and PI-RADS ≥ 4 lesions were substantial(AC1=0.601) and almost perfect (PA=0.876), respectively. Conclusion: Readers achieved moderate agreement for PI-RADS version 2.1 and benefitted from training sessions. Feedback, training, and multidisciplinary discussions also improved inter-reader agreement. Our study can provide guidance, updates, and further steps for the standardization and improvement of PI-RADS scoring.