0

MATH: Measuring Mathematical Problem Solving

Active

Dataset of 12,500 challenging competition mathematics problems. Demonstrates fewshot prompting and custom scorers. NOTE: The dataset has been taken down due to a DMCA notice from The Art of Problem Solving.

Open
Publisher
University of California, Berkeley
Domain
Mathematics
License
mit
Published
May 2026
Notable for
Benchmark for evaluating Mathematics.
Canonical
github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/math

Cite

Notes

Only stored in your browser.

Attribution

Attribution policy →

Related tools

4
View all

Implementations, trainers, datasets and scaffolds linked to this eval.

Hendrycks MATH RL Env (Community)

MATH (Hendrycks) evaluation matching OLMES minerva_math::tulu methodology

ImplementationRL EnvMath

Hendrycks MATH RL Env (Prime Intellect)

Prime Intellect

Single-turn Hendrycks MATH-style problems with boxed numeric answers and CoT.

ImplementationRL EnvMath

Hendrycksmath RL Env (Community)

hendrycksmath evaluation environment

Trains towardRL EnvMath

MATH 500 RL Env (Prime Intellect)

Prime Intellect

MATH-500 evaluation environment

Trains towardRL EnvMath

Papers

1

Measuring Mathematical Problem Solving With the MATH Dataset

NeurIPS · 2021

Introduces the MATH benchmark of 12,500 competition-level math problems with step-by-step solutions, spanning algebra to number theory at high-school olympiad difficulty.

FAQ

What is MATH: Measuring Mathematical Problem Solving?
Dataset of 12,500 challenging competition mathematics problems. Demonstrates fewshot prompting and custom scorers. NOTE: The dataset has been taken down due to a DMCA notice from The Art of Problem Solving.
How can a model improve its MATH: Measuring Mathematical Problem Solving score?
Tools linked to MATH: Measuring Mathematical Problem Solving on Sophon include Hendrycks MATH RL Env (Community), Hendrycks MATH RL Env (Prime Intellect), Hendrycksmath RL Env (Community), MATH 500 RL Env (Prime Intellect) - RL environments, datasets, and scaffolds that target this eval.
What license is MATH: Measuring Mathematical Problem Solving under?
MATH: Measuring Mathematical Problem Solving is available under mit.