Diagnostic Accuracy and Reproducibility of ChatGPT-4o for HER2 Immunohistochemistry Scoring in Equivocal…

AI / informaticsAsian Pacific journal of cancer prevention : APJCP· May 2026

Diagnostic Accuracy and Reproducibility of ChatGPT-4o for HER2 Immunohistochemistry Scoring in Equivocal Breast Cancer Cases.

Cheep Charoenlap, Pakorn Arunsawat, Kittiphan Chienwichai … Arunchai Chang

Breast & Gynecologic PathologyAI / informaticsDiagnosisQ3IF 1.9

ChatGPT-4o had only 30% sensitivity for detecting HER2-positive equivocal breast cancer cases.

Diagnostic Accuracy and Reproducibility of ChatGPT-4o for HER2 Immunohistochemistry Scoring in Equivocal Breast Cancer Cases.

Cheep Charoenlap … Arunchai Chang

Asian Pacific journal of cancer prevention · May 2026

Background

HER2 immunohistochemistry (immunohistochemistry) plays a central role in therapeutic decision-making for breast cancer.

Purpose

This study evaluated the diagnostic performance and reproducibility of ChatGPT-4o, a general-purpose large language model, in scoring HER2 immunohistochemistry in breast cancer cases initially classified as immunohistochemistry 2+.

Methods

We retrospectively analyzed 81 formalin-fixed, paraffin-embedded invasive carcinoma of no special type (no special type) cases with prior HER2 immunohistochemistry 2+ scores and corresponding dual in situ hybridization (DISH) results.

Results

chatgpt missed most true HER2-positive cases, flagging them as negative

30%

Sensitivity

100%

Specificity

More results

Fourteen cases (17.3%) remained equivocal following AI interpretation.

Intra-model reproducibility was good (intraclass correlation coefficient = 0.78), whereas agreement with HER2 DISH was fair (Cohen's κ = 0.375).

Misclassification predominantly involved false-negative interpretations among HER2-positive cases.

“

Conclusion · 1 of 2

ChatGPT-4o demonstrated high specificity and reproducibility for identifying HER2 immunohistochemistry 3+ cases but showed limited sensitivity and only fair concordance with HER2 DISH.

Conclusion · 2 of 2

These findings indicate that, in its current general-purpose form, ChatGPT-4o is not suitable for independent HER2 assessment and may serve, at best, as an exploratory adjunct to pathologist interpretation.

Diagnostic Accuracy and Reproducibility of ChatGPT-4o for HER2 Immunohistochemistry Scoring in Equivocal Breast Cancer Cases.

Diagnostic Accuracy and Reproducibility of ChatGPT-4o for HER2 Immunohistochemistry Scoring in Equivocal Breast Cancer Cases.

Related papers