AIME 2024: Problems from the American Invitational Mathematics Examination

AIME-24 evaluation environment

Hermes Example RL Env (Community) is an RL environment, used to improve scores on AIME 2024: Problems from the American Invitational Mathematics Examination.

Multi-turn math problem-solving environment where the model proposes Python code in a sandbox to compute and verify numerical answers.

Training environment for DeepScaleR, containing single-turn math problems with rule-based grading

Single-turn DeepScaleR math problems with boxed numeric answers and CoT.

Environment for single-turn tasks in OpenBench

DeepConf environment for confidence-aware LLM reasoning evaluation

An 860k-problem competition-math dataset with detailed solutions, the open community's go-to corpus for training math-specialized LLMs.

A fully-open distillation of long DeepSeek-R1 reasoning traces - the community's flagship "open R1" SFT corpus for reasoning models.

Stanford's hand-curated 1,000-problem reasoning dataset that, paired with budget forcing at inference, produced o1-competitive results for ~$50 of compute.

Convention of using the American Invitational Mathematics Examination (AIME) - 15 hard math problems per year - to evaluate frontier reasoning models.

Convention of using the American Invitational Mathematics Examination (AIME) - 15 hard math problems per year - to evaluate frontier reasoning models.