prolog

Procedurally generated SWI-Prolog constraint-satisfaction tasks. An agent edits and runs a starter solution.pl in a sandbox to solve one of nine task kinds across three difficulties; the reward runs the solve/1 query in the same sandbox and verifies the answer in Python against held-out generator metadata, so any valid solution is accepted and editing the facts cannot spoof reward.

Taskset

• Source: Procedurally generated — nine task kinds (sudoku, graph_coloring, zebra, nqueens, cryptarithm, scheduling, nonogram, bin_packing, hamiltonian) across three difficulties (medium, hard, expert)
• Size: procedurally generated; count configurable per eval (default pool of 100 tasks)

Notes

• Requires a container sandbox with SWI-Prolog (swipl:latest); the task sets NEEDS_CONTAINER, so the local subprocess runtime is refused.
• Grading integrity: the EXIT_CODE:N marker appended by the verification command is parsed from its last occurrence, so a solve/1 that prints a fake EXIT_CODE:0 and then fails cannot spoof success.
• scheduling targets are provably-optimal makespans and bin_packing instances are perfect packings (total size = bins × capacity), so greedy heuristics (e.g. first-fit-decreasing) provably fail — only genuine constraint solving passes.

Changelog

• 2026-09-03: Restore default solver network access by reverting the network_allow=[] default-deny policy introduced in #780; training rollouts need outbound network.
• 2026-08-31: Validate task kind and difficulty in configuration, then yield procedurally generated tasks on demand.
• 2026-08-02: Renamed the inline prompt option to --env.taskset.task-system-prompt; --env.taskset.system-prompt remains the framework's prompt-file override.
• 2026-07-10: Ported to the task-centric verifiers API: rewards and lifecycle hooks live on the Task (a TaskData row + behavior split), and task-facing config knobs (judges, tool/user placement, scoring parameters) moved from --env.taskset.* to --env.taskset.task.*. Requires verifiers>=0.2.0 and Python >=3.11.