OfficeQA Pro V2
A re-implementation of OfficeQA Pro V2 using Verifiers v1.
Changes compared to upstream
- Basic blocklist of domains where the dataset is hosted to avoid simple lookups (Hugging Face, GitHub, and common mirrors).
- Changed the regex-based grader to an LLM-as-judge to catch semantic nuances better. We found this to be less brittle. The judge is GPT-5.6 Luna @ medium.
Set env.taskset.network_block = [] to allow all domains for an evaluation. Omitting
this setting preserves the default dataset-host blocklist. For hosted evaluations,
pass it through --env-args '{"taskset":{"network_block":[]}}'.