Evaluation of AI Summaries on Interdisciplinary Understanding of Ophthalmology Notes.
Most non-eye-specialist clinicians preferred notes with an AI-generated plain-language summary added.
Evaluation of AI Summaries on Interdisciplinary Understanding of Ophthalmology Notes.
IMPORTANCE: Specialized ophthalmology terminology limits comprehension for nonophthalmology clinicians and professionals, hindering interdisciplinary communication and patient care.
To evaluate large language models-generated plain language summaries (plain language summaries) integrated into standard ophthalmology notes (standard ophthalmology notes) in improving diagnostic understanding, satisfaction, and clarity.
DESIGN, SETTING, AND PARTICIPANTS: Randomized quality improvement study conducted from February 1, 2024, to May 31, 2024, including data from inpatient and outpatient encounters in a single tertiary academic center.
most non-eye doctors preferred notes with the plain-language summary added
Plain language summaries semantic analysis found high meaning preservation (bidirectional encoder representations from transformers score mean F1 score: 0.85) with greater readability than standard ophthalmology notes (Flesch Reading Ease: 51.8 vs 43.6; Flesch-Kincaid Grade Level: 10.7 vs 11.9).
Ophthalmologists (n = 489; 84% response rate) reported high plain language summaries accuracy (90% [320 of 355] a great deal) with minimal review time burden (94.9% [464 of 489] ≤1 minute).
Plain language summaries error rate on ophthalmologist review was 26% (126 of 489).
In this study, use of large language models-generated plain language summaries was associated with enhanced comprehension and satisfaction among nonophthalmology clinicians and professionals, which might aid interdisciplinary communication.
Careful implementation and safety monitoring are recommended for clinical integration given the persistence of errors despite physician review.