swe-grep-env
Replace the placeholders below, then remove this callout.
Overview
- Environment ID:
swe-grep-env - Short description:
- Tags:
Datasets
- Primary dataset(s): <name(s) and brief description>
- Source links:
- Split sizes: <train/eval counts>
Task
- Type: <single-turn | multi-turn | tool use>
- Parser: <e.g., ThinkParser, XMLParser, custom>
- Rubric overview:
Quickstart
Run an evaluation with default settings:
uv run vf-eval swe-grep-env
Configure model and sampling:
uv run vf-eval swe-grep-env -m gpt-4.1-mini -n 20 -r 3 -t 1024 -T 0.7 -a '{"key": "value"}' # env-specific args as JSON
Notes:
- Use
-a/--env-argsto pass environment-specific configuration as a JSON object.
Environment Arguments
Document any supported environment arguments and their meaning. Example:
| Arg | Type | Default | Description |
|---|---|---|---|
foo | str | "bar" | What this controls |
max_examples | int | -1 | Limit on dataset size (use -1 for all) |
Metrics
Summarize key metrics your rubric emits and how they’re interpreted.
| Metric | Meaning |
|---|---|
reward | Main scalar reward (weighted sum of criteria) |
accuracy | Exact match on target answer |