Impact of a machine learning-powered algorithm on pathologist HER2 IHC scoring in breast cancer
Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Background
HER2 expression level is a key factor in determining the optimal treatment course for breast cancer patients. Roughly 15% of breast cancers are HER2(+), and determination of HER2 status is routinely assessed by immunohistochemistry (IHC). Accurate assessment of the HER2 IHC score by pathologists is therefore critical, especially in light of novel therapeutic approaches demonstrating efficacy in the HER2-low setting. However, there is an opportunity to improve inter-pathologist agreement at the lower levels of HER2 scoring (0, 1+, and 2+).
Methods
A machine learning model (AIM-HER2) was developed to generate accurate, slide-level HER2 scores aligned with ASCO-CAP guidelines in clinical breast cancer HER2 IHC specimens. AIM-HER2 was assessed as an AI-assist tool in a retrospective reader study, where 20 HER2-trained pathologists scored breast cancer cases (N=200) with and without AIM-HER2 assistance using a 2-cohort crossover design with a 3-week washout. A separate panel of 5 expert HER2 pathologists read all 200 cases manually to establish reference scores.
Results
In a significant fraction of cases examined, less than 70% inter-pathologist agreement was observed. When used as an AI assist tool, AIM-HER2 improved inter-rater agreement overall and specifically at the 0/1+ and 1+/2+ cutoffs. Similarly, AIM-HER2 AI-assist significantly increased PPA at the 0/1+ and 1+/2+ cutoffs. When interacting with the AI-assist tool, pathologists displayed a wide range of override rates, and the quality of a pathologist’s overrides was correlated with their manual accuracy. Lastly, the impact of the reference panel on AIM-HER2 accuracy metrics was assessed, revealing that measurements of model accuracy are highly dependent on reference panel composition.
Conclusions
The use of AIM-HER2 as an AI-assist tool for scoring HER2 IHC in breast cancer may improve pathologist reproducibility and accuracy, particularly at the 0/1+ and 1+/2+ cutoffs.
