hallucination
Artificial Analysis
Artificial Analysis's broad-knowledge benchmark - thousands of curated factual questions spanning specialized domains - designed to test hallucination calibration.
Stanford Center for Research on Foundation Models (CRFM)
Stanford CRFM's wide-coverage evaluation framework - dozens of scenarios scored on accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency.
OpenAI
4,326 short factual questions with a single unambiguous answer, designed to measure short-form factuality and calibrate hallucination rate.
Future of Humanity Institute (Oxford)
817 questions targeting common human misconceptions, measuring whether a model gives factually true answers or repeats popular falsehoods.
Vals AI
Vals.ai's proprietary suite of legal-domain benchmarks (contract review, hallucination tests, LegalBench Pro) used by law firms to procure LLMs.
Anthropic
Anthropic's foundational helpful-and-harmless human preference dataset - the first major public RLHF corpus and a long-time community baseline.
Prime Intellect
Multi-turn open-web research QA environment where the model uses a Serper search tool to answer hard, hop-heavy questions of the BrowseComp / SimpleQA style.
SimpleQA evaluation environment
SimpleQA eval
SimpleQA-Verified evaluation environment with accuracy scoring.
Environment for single-turn tasks in OpenBench
by avg parsed score across evals here
