How can a model improve its MATH: Measuring Mathematical Problem Solving score?

Tools linked to MATH: Measuring Mathematical Problem Solving on Sophon include Hendrycks Math (Pmahdavi), Hendrycks Math (Prime Intellect), Hendrycksmath, Math500 - RL environments, datasets, and scaffolds that target this eval.

What license is MATH: Measuring Mathematical Problem Solving under?

MATH: Measuring Mathematical Problem Solving is available under mit.

MATH: Measuring Mathematical Problem Solving

Active

Dataset of 12,500 challenging competition mathematics problems. Demonstrates fewshot prompting and custom scorers. NOTE: The dataset has been taken down due to a DMCA notice from The Art of Problem Solving.

Open

Publisher: University of California, Berkeley
Domain: Mathematics
License: mit
Published: May 2026
Notable for: Benchmark for evaluating Mathematics.
Canonical: github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/math

Cite

Notes

Only stored in your browser.

Attribution

README: github.com/UKGovernmentBEIS/inspect_evals/blob/main/src/inspect_evals/math/README.mdMIT

Attribution policy →

Related tools

View all

Implementations, trainers, datasets and scaffolds linked to this eval.

Hendrycks Math (Pmahdavi)

MATH (Hendrycks) evaluation matching OLMES minerva_math::tulu methodology

ImplementationRL EnvMath

Hendrycks Math (Prime Intellect)

Prime Intellect

Single-turn Hendrycks MATH-style problems with boxed numeric answers and CoT.

ImplementationRL EnvMath

Hendrycksmath

hendrycksmath evaluation environment

Trains towardRL EnvMath

Math500

Prime Intellect

MATH-500 evaluation environment

Trains towardRL EnvMathV1

Papers

Measuring Mathematical Problem Solving With the MATH Dataset

NeurIPS · 2021

Introduces the MATH benchmark of 12,500 competition-level math problems with step-by-step solutions, spanning algebra to number theory at high-school olympiad difficulty.

FAQ

What is MATH: Measuring Mathematical Problem Solving?: Dataset of 12,500 challenging competition mathematics problems. Demonstrates fewshot prompting and custom scorers. NOTE: The dataset has been taken down due to a DMCA notice from The Art of Problem Solving.
How can a model improve its MATH: Measuring Mathematical Problem Solving score?: Tools linked to MATH: Measuring Mathematical Problem Solving on Sophon include Hendrycks Math (Pmahdavi), Hendrycks Math (Prime Intellect), Hendrycksmath, Math500 - RL environments, datasets, and scaffolds that target this eval.
What license is MATH: Measuring Mathematical Problem Solving under?: MATH: Measuring Mathematical Problem Solving is available under mit.