Reference Hallucination in AI-Assisted Academic Writing: A Comparative Analysis of ChatGPT, Gemini, and…

AI / informaticsIndian journal of orthopaedics· Aug 2026

Reference Hallucination in AI-Assisted Academic Writing: A Comparative Analysis of ChatGPT, Gemini, and Perplexity in Rotator Cuff Literature.

İlhan Celil Özbek, Fatih Bağcıer

Shoulder & ElbowAI / informaticsClinical practiceQ3IF 1.6

Perplexity generated the most hallucinated references among three AI chatbots tested.

Reference Hallucination in AI-Assisted Academic Writing: A Comparative Analysis of ChatGPT, Gemini, and Perplexity in Rotator Cuff Literature.

İlhan Celil Özbek, Fatih Bağcıer

Indian journal of orthopaedics · Aug 2026 · Q3 in Orthopedics

Purpose

This study aimed to compare the reliability of references generated by three different AI-based conversational agents (ChatGPT, Gemini, Perplexity) within the rotator cuff literature.

Methods

Designed as a cross-sectional study, 30 subtopics were posed to each bot in two formats (letter to the editor and original article), resulting in a total of 3150 references.

n = 3150 references

Results

Perplexity fabricated references far more often, the worst of the three chatbots

1.81mean R

ChatGPT

4.01mean R

Gemini

6.51mean R

Perplexity

More results

In the letter format, ChatGPT, Gemini, and Perplexity scored 1.81 ± 3.20, 3.81 ± 4.83, and 6.43 ± 4.78, respectively.

In the article format, ChatGPT scored 4.02 ± 1.70, Gemini 4.13 ± 1.57, and Perplexity 6.31 ± 1.68.

ChatGPT was more reliable than Gemini and significantly superior to Perplexity.

“

Conclusion · 1 of 4

AI-based conversational agents may play a supportive role in academic writing; however, they exhibit critical limitations in reference accuracy.

Conclusion · 2 of 4

While ChatGPT was relatively the most reliable model, Perplexity had the highest hallucination rate.

Conclusion · 3 of 4

Researchers must refrain from using references generated by these tools without verification, as doing so poses significant concerns for scientific integrity and ethics.

Conclusion · 4 of 4

SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s43465-026-01807-0.

Reference Hallucination in AI-Assisted Academic Writing: A Comparative Analysis of ChatGPT, Gemini, and Perplexity in Rotator Cuff Literature.

Reference Hallucination in AI-Assisted Academic Writing: A Comparative Analysis of ChatGPT, Gemini, and Perplexity in Rotator Cuff Literature.

Related papers