0

Navital Upi

Fresh

UPI Desk (Navital Gym v1.1.0): 50 public, deterministic, rule-graded UPI payment and dispute tasks for agents. No AI judge. Public split only.

Type
RL Env
Publisher
Navitallabs
Runtime
multi-turn
License
apache-2.0
Size
v0.1.2
Published
Nov 2026
Updated
Nov 2026

Cite

Notes

Only stored in your browser.

navital-upi

UPI Desk, one of six worlds in Navital Gym v1.1.0, as a verifiers environment. By Navital Labs.

  • What: 50 public tasks. An agent handles UPI payment attempts and disputes in a sandboxed bank under written, simplified rules: pay and verify, limits, failed-but-debited, wrong recipient, collect requests (phased out 28 Feb 2026), mandates, UPI Lite. The world has 18 tools; creating a mandate always fails by design (registration needs the owner's own PIN).
  • Grading: deterministic rules, never an AI judge. Reward = the world's own grader (score = fraction of rules passed, in [0, 1]) run on the last JSON object in the final message and on the tool calls the rollout really executed. solved (all required rules passed, no forbidden step) is a second metric. A forbidden step counts even if the simulated bank refused it. 29 of the 50 tasks have a forbidden step.
  • Hidden fields (expected answers, traps) never reach the agent or the traces, as long as the agent only has the world's tools; they are read host-side at scoring time. A runtime that gives the agent a shell or file access (for example a hosted sandbox with a shell) has not been tested.
  • Public only: these are the same public tasks as the open Gym. Models may have seen them. Our private held-out sets are NOT in this package.
  • Synthetic data. Every world is a simplified, documented version of Indian practice, not a statement about current law, bank or NPCI rules. Not affiliated with NPCI or any bank.

Use

prime env install navitallabs/navital-upi          # once published
vf-eval navital-upi -m <model> -n 50
import verifiers.v1 as vf
from navital_upi import load_environment
env = load_environment()                 # all 50 tasks; ids=["IN-UPI-001"] for a subset

Runtime (read this): the per-rollout tool server needs navital-gym on the machine that runs it, so use a local runtime, for example:

vf-eval navital-upi -m <model> --env.agent.runtime.type subprocess --env.agent.harness.id null

Prime's default hosted sandbox runtime (a bare python:3.11-slim image) has NOT been tested with this package and is not expected to work without navital-gym in the image.

Needs navital-gym==1.1.0 importable (it is a dependency). The per-rollout tool server runs as a local subprocess (python -m navital_gym.adapters.verifiers_env).

Honest limits

Written against verifiers 0.4.0's v1 API (pinned), which is young. The adapter has been tested offline with a scripted model and by a reward-parity test (reward and solved equal the Gym's own grader on 250 recorded real rollouts, 5 models x 50 tasks, 0 mismatches; a control that drops the tool results changes the reward on 36 of 50 tasks, so the test can fail); it has not been run on Prime's hosted evals.

Results

Reference results are kept on the public leaderboard: https://navitallabs.com/labs/navital-gym (one attempt per task; labelled by engine; public tasks).

Licence

Code: Apache-2.0 (LICENSE). Task data: CC BY 4.0, attribution to Navital Labs / Navital Gym.