Evaluating Large Language Models on Aerospace Medicine Principles.

AI / informaticsWilderness & environmental medicine· Sep 2025

Evaluating Large Language Models on Aerospace Medicine Principles.

Kyle D Anderson, Cole A Davis, Shawn M Pickett, Michael S Pohlen

Read paper

Aerospace MedicineAI / informaticsQ3IF 1.4

Custom Retrieval-Augmented Generation-based large language models answered aerospace medicine board questions most accurately

Evaluating Large Language Models on Aerospace Medicine Principles.

Kyle D Anderson … Michael S Pohlen

Wilderness & environmental medicine · Sep 2025 · Q3 in Public Health

Background

Large language models (large language models) hold immense potential to serve as clinical decision-support tools for Earth-independent medical operations.

Methods

To better understand this risk, this work tested two publicly available large language models, ChatGPT-4 and Google Gemini Advanced (1.0 Ultra), as well as a custom Retrieval-Augmented Generation (Retrieval-Augmented Generation) large language models on factual knowledge and clinical reasoning in accordance with published material in aerospace medicine.

n = 857 free-response questions

Results

custom Retrieval-Augmented Generation tool answered the most board questions correctly, beating both public chatbots

70%

ChatGPT-4

55%

Gemini Advanced

85%

RAG LLM

More results

When queried with 857 free-response questions from Aerospace Medicine Boards Questions and Answers , ChatGPT-4 had a mean reader score from 4.23 to 5.00 (Likert scale of 1-5) across chapters, whereas Gemini Advanced and the Retrieval-Augmented Generation large language models scored 3.30 to 4.91 and 4.69 to 5.00, respectively.

“

Conclusion

There is considerable promise for large language models use in autonomous medical operations in spaceflight given the anticipated continued rapid pace of development, including advancements in model training, data quality, and fine-tuning methods.

Evaluating Large Language Models on Aerospace Medicine Principles.

Evaluating Large Language Models on Aerospace Medicine Principles.

Related papers