clinical-diagnosis-assistant

This environment tests an agent's ability to diagnose medical conditions by iteratively gathering patient information, ordering diagnostic tests, and consulting medical knowledge bases. The agent must manage patient state and tool usage to arrive at an accurate diagnosis.

Overview

Domain: medicine Base Class: StatefulToolEnv Difficulty: medium Task: The model must interact with a simulated patient, ask relevant questions, order appropriate diagnostic tests, and use medical reference tools to determine the correct diagnosis for a given set of symptoms.

Quickstart

Installation

uv run vf-install clinical-diagnosis-assistant

Usage

import verifiers as vf

env = vf.load_environment("clinical-diagnosis-assistant")
results = env.evaluate_sync(
    client=vf.OpenAI(),
    model="gpt-4.1-mini",
    num_examples=10,
    rollouts_per_example=1
)

Evaluation

Run an evaluation with default settings:

uv run vf-eval clinical-diagnosis-assistant

Configure model and sampling:

uv run vf-eval clinical-diagnosis-assistant \
  -m gpt-4.1-mini \
  -n 20 -r 3 -t 1024 -T 0.7

Environment Arguments

Arg	Type	Default	Description
`num_examples`	int	1000	Number of training examples
`num_eval_examples`	int	100	Number of evaluation examples
`seed`	int	42	Random seed for reproducibility

Metrics

Metric	Meaning
`reward`	Primary reward signal
`format_reward`	Format adherence reward (if applicable)

About

Generated by synthetic-rl-env-creator.

Tags: medicine