longcot-mini
LongCoT-mini long-horizon reasoning tasks spanning five domains (logic, cs, chemistry, chess, math), each a self-contained multi-step task solved by an agent in a sandbox. The agent writes its final answer to /workspace/answer.txt and is scored against gold by the upstream longcot.verify template dispatch (plus a local math numeric-equivalence fallback), yielding a rule-based correct reward.
Taskset
- Source: LongHorizonReasoning/longcot
- Size: ~500 tasks (the upstream
easydifficulty split across all five domains, minus 21 broken easy-math IDs excluded by default)
Changelog
- 2026-08-31: Yield task records on demand so bounded evaluations construct only the requested prefix.
- 2026-06-24: Initial v1 taskset.