What capabilities does SWE-Gym test?

SWE-Gym evaluates code editing, debugging, tool calling, planning.

What license is SWE-Gym under?

SWE-Gym is available under MIT.

SWE-Gym

Active

First publicly available training environment for software-engineering agents - 2,438 real Python GitHub issues with executable Docker test environments and golden patches.

Open

Publisher: University of California, Berkeley
Capabilities: Code Editing Debugging Tool Calling Planning
Domain: code
Format: Custom
Size: 2,438 instances (Lite: 230) tasks
License: MIT
Published: Dec 2024
Notable for: Benchmark for evaluating code editing, debugging and tool calling in the code domain.
Canonical: github.com/SWE-Gym/SWE-Gym

Cite

Notes

Only stored in your browser.

Papers

SWE-Gym: An Open Environment for Training Software Engineering Agents and Verifiers

preprint · 2024

UC Berkeley + UIUC open environment of 2,438 Python repo issues with executable tests for training and verifying SWE-bench-style agents.

introduces

SWE-Gym: An Open Environment for Training Software Engineering Agents and Verifiers

preprint · 2024

UC Berkeley + UIUC open environment of 2,438 Python repo issues with executable tests for training and verifying SWE-bench-style agents.

FAQ

What is SWE-Gym?: First publicly available training environment for software-engineering agents - 2,438 real Python GitHub issues with executable Docker test environments and golden patches.
What capabilities does SWE-Gym test?: SWE-Gym evaluates code editing, debugging, tool calling, planning.
What license is SWE-Gym under?: SWE-Gym is available under MIT.