Post

New research · Ophthalmology
Veterinary ophthalmology · 22h
Animal / preclinicalVeterinary ophthalmology · 2026

Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology.

Malwina Ewa Kowalska, Niklas Holz, Simon Anton Pot … Sonja Hartnack
Read paper
OphthalmologyAnimal / preclinical

Balancing data distribution substantially increased Cohen's kappa agreement for left eyes.

Challenges and Misinterpretations of Cohen's Kappa in Agreement Studies in Ophthalmology.

Malwina Ewa Kowalska et al. · Veterinary ophthalmology · 2026
Background

Cohen's kappa is widely used to measure rater agreement on binary outcomes and performs well when outcome prevalence is relatively balanced.

Purpose

This paper aims to explain the challenges encountered with Cohen's kappa, and to provide guidance to veterinary ophthalmologists on the usage and interpretation of kappa values.

Methods

Following the 2022 ECVO-HED (European College of Veterinary Ophthalmology-Hereditary Eye Diseases) gonioscopy grading scheme and its general recommendations, eyes were classified into two categories-"breeding-YES" (not affected, mildly or moderately affected) or "breeding-NO" (severely affected).

n = 60 eyes
0.77
Results
This shows higher agreement between examiners on left eye condition after balancing the data
n = 60 eyes
More results

While the two examiners classified 52/60 left and 51/58 right eyes into the "breeding-YES" category, they disagreed on 7/60 left and 1/58 right eyes, with resulting kappa values of 0.18 and 0.91 for the left and right eyes, respectively.

“
Conclusion · 1 of 2

Readers should be cautious when comparing kappa values across studies with different underlying prevalence and bias.

Conclusion · 2 of 2

Under non-ideal data distribution, additional statistical indices like prevalence- and bias index, prevalence-and-bias-adjusted kappa (PABAK), and maxKappa, do not replace Cohen's kappa, but assist in interpreting it.

Read paper
0 comments

No comments yet. Be the first.

Related papers

LatestFoundational
Cohort Study
median 9
optometry referrals contained more complete documentation for glaucoma
Study
71.27%
medical students' higher accuracy in identifying eye infections after artificial intelligence training
Cross-sectional
64%
most patients reported little difficulty with their eye injections
Cross-sectional
37%
Only 37% of primary care providers and endocrinologists correctly identified diabetic retinopathy status.
Randomized Trial
adjusted difference -4.97
Baduanjin exercise led to a better overall quality of life than routine care
Study
58.26%
Qwen-7B correctly found eye diseases in over half of cases without specific training
Case Report
24 months
the patient's eye cancer remained gone for this period after treatment
Study
Mean 184 vs. 137
Residents in subsidized programs performed more cataract surgeries as primary surgeon.