Benchmarking large language models against practicing clinicians on psychopathological assessment.

AI / informaticsNPJ digital medicine· Jul 2026

Benchmarking large language models against practicing clinicians on psychopathological assessment.

Esra Lenz, Joonas Naamanka, Wolfgang Trabert … Emanuel Schwarz

Read paper

Mood DisordersAI / informaticsQ1IF 18.0

Top LLMs achieved 0.72 accuracy in psychiatric symptom assessment.

Benchmarking large language models against practicing clinicians on psychopathological assessment.

Esra Lenz … Emanuel Schwarz

NPJ digital medicine · Jul 2026 · Q1 in General Medicine

Background

Psychiatry's reliance on language makes LLMs a natural tool for psychopathological assessment, yet structured, item-level assessments from psychiatric clinical interviews remain under-researched.

Methods

These proof-of-concept findings require validation in real patient interviews, larger samples, and prospective studies integrating multimodal input.

0.72

Results

0.72

This accuracy placed top LLMs at the 64th percentile of clinician performance.

More results

In this proof-of-concept study, 10 LLMs assessed transcripts of three simulated psychiatric interviews across all 100 items of the Association for Methodology and Documentation in Psychiatry (Association for Methodology and Documentation in Psychiatry) system, benchmarked against 108 early-career clinicians rating full audiovisual recordings, using an expert consensus panel as reference.

More results

GPT-5.1, selected for a marginal advantage, showed per-scenario accuracies of 0.81 (depression), 0.76 (mania), and 0.60 (schizophrenia) versus clinician means of 0.79, 0.68, and 0.58.

“

Conclusion

These proof-of-concept findings require validation in real patient interviews, larger samples, and prospective studies integrating multimodal input.

Benchmarking large language models against practicing clinicians on psychopathological assessment.

Benchmarking large language models against practicing clinicians on psychopathological assessment.

Related papers