Rebecca Attal - research works

GPT-4.0 demonstrated moderate performance (2.71 ± 1.46 overall), while Gemini 2.0 scored lowest (2.14 ± 1.01).

Mixed-effects modeling demonstrated significant group differences across all domains ( p ≤ 0.008).

Citation quality showed one of the largest between-group differences and strong reliability [ICC(2,1) = 0.68; ICC(2,7) = 0.94].

Manual verification of 123 AI-generated references revealed high citation accuracy for GPT-4.0 (89.6% fully accurate; 0% fabricated) compared with Gemini 2.0 (71.7% fully accurate; 23.9% fabricated).

Human-written abstracts were more often misclassified as AI than AI-generated abstracts were mistaken for human.

Human abstracts received significantly higher clarity and usefulness scores than LLM abstracts, though effect sizes were small.

Confidence did not correlate with correctness, indicating miscalibration of rater judgments.

Grok-generated abstracts were most easily identified as AI, whereas GPT-5 and Claude 3.5 more frequently resembled human writing.

Mean lay ratings for clarity across AI-generated forms were high overall.

However, when analyzed at the form level, differences between models were not statistically significant for clarity (mean difference 0.04; t (9) = -0.80; p = 0.44) or signing comfort (mean difference 0.13, t (9) = -1.68, p = 0.13).

Mean Flesch-Kincaid Grade Level was lower for AI-generated forms compared to official templates.

Although prompts targeted a 6th-8th grade reading level, achieved readability scores were slightly higher (8.8-9.4).