Language models hallucinate partly because training and benchmarks reward confident guessing over admitting uncertainty.

Is it true that language models hallucinate partly because training and benchmarks reward confident guessing over admitting uncertainty?

Yes. Two sources directly support the claim: an arXiv paper arguing that training and evaluation procedures reward guessing over acknowledging uncertainty, and a peer-reviewed Nature paper showing accuracy-based evaluations inadvertently reward unwarranted guessing. Two further items support only the training-reward half, noting supervised fine-tuning implicitly rewards always responding, without addressing benchmarks.

Verdict: SUPPORTED · 4 sources for, 0 against · Last checked 3 Oct 2026

The private library was unavailable for some searches in this check. Missing library results are not evidence against the claim.

[2509.04664] Why Language Models Hallucinate. (n.d.). arxiv.org. https://arxiv.org/abs/2509.04664

AT, K., O, N., SS, V., & E, Z. (2026). Evaluating large language models for accuracy incentivizes hallucinations.. doi.org. https://doi.org/10.1038/s41586-026-10549-w

HypoTermInstruct: Instructing Large Language Models not to Hallucinate. (n.d.). openreview.net. https://openreview.net/forum?id=pfmacmILs2

Inducing Epistemological Humility in Large Language Models: A Targeted SFT Approach to Reducing Hallucination. (2026). doi.org. https://doi.org/10.48550/arxiv.2603.17504