TruthfulQA: Measuring How Models Mimic Human Falsehoods

Active

Measure whether a language model is truthful in generating answers to questions using questions that some humans would answer falsely due to a false belief or misconception.

Open

Publisher: University of Oxford
Domain: Knowledge
License: mit
Published: May 2026
Notable for: Benchmark for evaluating Knowledge.
Canonical: github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/truthfulqa

Cite

Notes

Only stored in your browser.

Attribution

README: github.com/UKGovernmentBEIS/inspect_evals/blob/main/src/inspect_evals/truthfulqa/eval.yamlMIT

Attribution policy →

Papers

TruthfulQA: Measuring How Models Mimic Human Falsehoods

ACL · 2021

Introduces TruthfulQA, 817 adversarial questions designed so that imitating common human misconceptions yields wrong answers; larger models often do worse.

FAQ

What is TruthfulQA: Measuring How Models Mimic Human Falsehoods?: Measure whether a language model is truthful in generating answers to questions using questions that some humans would answer falsely due to a false belief or misconception.
What license is TruthfulQA: Measuring How Models Mimic Human Falsehoods under?: TruthfulQA: Measuring How Models Mimic Human Falsehoods is available under mit.