Sholem Hack - research works

GPT-4.0 demonstrated moderate performance (2.71 ± 1.46 overall), while Gemini 2.0 scored lowest (2.14 ± 1.01).

Mixed-effects modeling demonstrated significant group differences across all domains ( p ≤ 0.008).

Citation quality showed one of the largest between-group differences and strong reliability [ICC(2,1) = 0.68; ICC(2,7) = 0.94].

Manual verification of 123 AI-generated references revealed high citation accuracy for GPT-4.0 (89.6% fully accurate; 0% fabricated) compared with Gemini 2.0 (71.7% fully accurate; 23.9% fabricated).

Human-written abstracts were more often misclassified as AI than AI-generated abstracts were mistaken for human.

Human abstracts received significantly higher clarity and usefulness scores than LLM abstracts, though effect sizes were small.

Confidence did not correlate with correctness, indicating miscalibration of rater judgments.

Grok-generated abstracts were most easily identified as AI, whereas GPT-5 and Claude 3.5 more frequently resembled human writing.

Mean lay ratings for clarity across AI-generated forms were high overall.

However, when analyzed at the form level, differences between models were not statistically significant for clarity (mean difference 0.04; t (9) = -0.80; p = 0.44) or signing comfort (mean difference 0.13, t (9) = -1.68, p = 0.13).

Mean Flesch-Kincaid Grade Level was lower for AI-generated forms compared to official templates.

Although prompts targeted a 6th-8th grade reading level, achieved readability scores were slightly higher (8.8-9.4).

Twenty-four patients met inclusion criteria (mean age 45.2 years; 54.2% male).

All exhibited maxillary sinus atelectasis with orbital floor retraction.

No intraoperative complications or revision surgeries occurred.

Mean quality-of-life score was 4.58/5.

Revision occurred in 31 children: 6 (1.4%) in the microdebrider group and 25 (10.1%) in the coblation group.

After 36 months, revision-free survival was 99.4% for microdebrider and 96.1% for coblation (log-rank p < 0.001).

Age, sex, and tonsil size were not significant predictors.

Differences were attenuated when vignettes contained clearer diagnostic information.

Other effects were smaller, including increased neuroimaging recommendations for low-income descriptors (+4.0 pp; 95% CI 2.1-5.8).

ConclusionLLM clinical recommendations varied by sociodemographic descriptors, particularly under diagnostic uncertainty.