month_end_close_v1: month-end close pipeline

A "close" chains two to four finance tasks (invoice reconciliation, FIFO inventory, expense audit, timesheet pay) plus, from level 3, a final summary. Some inputs to each stage are not given directly: they're defined by small exact formulas over the exact results of earlier stages, so one wrong number upstream corrupts every stage that reads it. Every stage is graded exactly against the true chain, and a separate consistency score shows whether later stages are right given the model's own earlier answers.

Measured difficulty

With a bash tool (October 2026, 6 attempts per level, levels 9 and 10): Claude Sonnet 5.5 passed 12 of 12, while Claude Opus 5.5 passed only 3 of 12. Small samples, so treat the gap as a lead, not a result.

How it's graded

Exact and deterministic. Each instance is generated from a seed, and the verifier recomputes the correct answer from the instance itself and compares it field by field in exact arithmetic. There is no LLM judge and no stored answer key. Malformed or tampered answers score 0.

Install and run

pip install verifiers
vf-eval month_end_close_v1

Works on verifiers 0.1.x (load_environment) and >= 0.4 (verifiers.v1 Taskset). Seeds are public by default; pass secret= (0.1.x) or --env.taskset.secret (0.4) to generate private instances nobody else can reproduce.

About

Built by Clovrr Labs: exact-graded tasks for AI agents that write data. This is one of our baseline finance tasks, published as a free, open example. Our harder interactive tasks (ledger sync through an API that injects realistic faults, and a revenue report built from conflicting requirements) are not public: on ledger sync, Claude Sonnet 5.5 passes 25%, Claude Opus 5.5 38% and GPT-6.1 Sol 75%. Contact aryan@clovrr.net for access or custom batches.