contract-review-clause-identification

This environment tests an agent's ability to review legal contracts, identify specific clauses based on given criteria, and extract relevant information using a set of legal tools. It evaluates precision in clause identification and data extraction.

Overview

Domain: law Base Class: StatefulToolEnv Difficulty: medium Task: The model must review a provided contract, identify clauses matching a specified type (e.g., 'Force Majeure', 'Confidentiality'), and extract key details from those clauses using available tools.

Quickstart

Installation

uv run vf-install contract-review-clause-identification

Usage

import verifiers as vf

env = vf.load_environment("contract-review-clause-identification")
results = env.evaluate_sync(
    client=vf.OpenAI(),
    model="gpt-4.1-mini",
    num_examples=10,
    rollouts_per_example=1
)

Evaluation

Run an evaluation with default settings:

uv run vf-eval contract-review-clause-identification

Configure model and sampling:

uv run vf-eval contract-review-clause-identification \
  -m gpt-4.1-mini \
  -n 20 -r 3 -t 1024 -T 0.7

Environment Arguments

Arg	Type	Default	Description
`num_examples`	int	1000	Number of training examples
`num_eval_examples`	int	100	Number of evaluation examples
`seed`	int	42	Random seed for reproducibility

Metrics

Metric	Meaning
`reward`	Primary reward signal
`format_reward`	Format adherence reward (if applicable)

About

Generated by synthetic-rl-env-creator.

Tags: law