contract-clause-review

This environment tests an agent's ability to review legal contracts, identify specific clauses, and assess their compliance with predefined legal standards or client instructions. It requires sequential analysis and decision-making based on document content and legal tools.

Overview

Domain: law Base Class: StatefulToolEnv Difficulty: medium Task: The model must review a given contract, identify clauses related to a specific legal topic (e.g., liability, termination), and determine if they meet specified legal requirements or client preferences using provided legal tools.

Quickstart

Installation

uv run vf-install contract-clause-review

Usage

import verifiers as vf

env = vf.load_environment("contract-clause-review")
results = env.evaluate_sync(
    client=vf.OpenAI(),
    model="gpt-4.1-mini",
    num_examples=10,
    rollouts_per_example=1
)

Evaluation

Run an evaluation with default settings:

uv run vf-eval contract-clause-review

Configure model and sampling:

uv run vf-eval contract-clause-review \
  -m gpt-4.1-mini \
  -n 20 -r 3 -t 1024 -T 0.7

Environment Arguments

Arg	Type	Default	Description
`num_examples`	int	1000	Number of training examples
`num_eval_examples`	int	100	Number of evaluation examples
`seed`	int	42	Random seed for reproducibility

Metrics

Metric	Meaning
`reward`	Primary reward signal
`format_reward`	Format adherence reward (if applicable)

About

Generated by synthetic-rl-env-creator.

Tags: law