longcot_env
Long-horizon reasoning tasks spanning five domains (logic, cs, chemistry, chess, math) at medium and hard difficulty. Each self-contained prompt embeds the full task; the agent writes its final answer to /workspace/answer.txt, which is scored via the upstream longcot.verify template dispatch (with a local numeric-equivalence fallback for math templates).
Taskset
- Source: LongHorizonReasoning/longcot
- Size: ~2,000 tasks (medium + hard across all five domains), loaded from the bundled JSON in the
longcotpackage viaload_questions
Changelog
- 2026-09-03: Restore default solver network access by reverting the
network_allow=[]default-deny policy introduced in #780; training rollouts need outbound network. - 2026-08-31: Yield task records on demand so bounded evaluations construct only the requested prefix.
- 2026-06-24: Initial v1 taskset.