virl39k

ViRL39K — 38,870 verifiable vision-language QAs curated for RL training (VL-Rethinker, paper). Each task interleaves the question's images at its <image> markers and scores the final answer with math-verify.

Taskset

• Source: TIGER-Lab/ViRL39K (train split; images read lazily from the repo's images.zip, ~1.8 GB on first download)
• Prompt: the question with images interleaved, plus the upstream VL-Rethinker instruction: "Please reason step by step, and put your final answer within \boxed{}"
• Scoring: math-verify equivalence against the gold \boxed{} answer, lenient on format — the boxed answer is preferred, but without one math-verify's own extraction is used (LaTeX/plain expressions, plus option letters on its A–D default), so a correct answer is never scored 0 just for a missing box. has_boxed_answer is tracked as a metric.
• Filters: --env.taskset.category (exact match, e.g. Spatial Reasoning), --env.taskset.min-pass-rate / --env.taskset.max-pass-rate on the dataset's PassRate_32BTrained annotation for difficulty selection.

Deviations from upstream

• Upstream (VL-Rethinker) requires a \boxed{} answer; this port falls back to math-verify's own extraction when the box is missing, by design, to avoid penalizing format.
• ~850 rows carry no <image> marker in the question text; their image is placed before the question (the usual VLM convention).

Changelog

• 2026-09-03: Restore default solver network access by reverting the network_allow=[] default-deny policy introduced in #780; training rollouts need outbound network.
• 2026-07-19: Initial port.