How can a model improve its SWE-bench Verified: Resolving Real-World GitHub Issues score?

Tools linked to SWE-bench Verified: Resolving Real-World GitHub Issues on Sophon include Mini SWE Agent Bench (Prime Community), Deepswe (Prime Intellect), Mini SWE Agent Plus, Opencode SWE - RL environments, datasets, and scaffolds that target this eval.

What license is SWE-bench Verified: Resolving Real-World GitHub Issues under?

SWE-bench Verified: Resolving Real-World GitHub Issues is available under mit.

SWE-bench Verified: Resolving Real-World GitHub Issues

Active

Evaluates AI's ability to resolve genuine software engineering issues sourced from 12 popular Python GitHub repositories, reflecting realistic coding and debugging scenarios.

Open

Publisher: Princeton University
Domain: Coding
License: mit
Published: May 2026
Notable for: Benchmark for evaluating Coding.
Canonical: github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/swe_bench

Cite

Notes

Only stored in your browser.

Attribution

README: github.com/UKGovernmentBEIS/inspect_evals/blob/main/src/inspect_evals/swe_bench/README.mdMIT

Attribution policy →

Related tools

View all

Implementations, trainers, datasets and scaffolds linked to this eval.

Mini SWE Agent Bench (Prime Community)

Prime Community

Benchmarking model performance on SWE Bench in the Mini SWE Agent harness.

Trains towardRL EnvTool UseAgent

Deepswe (Prime Intellect)

Prime Intellect

DeepSWE environment for solving SWE issues inside Prime Sandboxes.

Trains towardRL EnvSWECode

Mini SWE Agent Plus

Prime Intellect

Mini SWE Agent Plus environment for solving SWE issues inside Prime Sandboxes.

Trains towardRL EnvSWECode

Opencode SWE

Prime Intellect

OpenCode SWE environment for solving SWE issues inside Prime Sandboxes.

Trains towardRL EnvSWECode

Papers

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

ICLR · 2024

Introduces SWE-bench, 2,294 real GitHub issues from 12 popular Python repos paired with their merged-PR test suites - a hard agentic coding benchmark.

FAQ

What is SWE-bench Verified: Resolving Real-World GitHub Issues?: Evaluates AI's ability to resolve genuine software engineering issues sourced from 12 popular Python GitHub repositories, reflecting realistic coding and debugging scenarios.
How can a model improve its SWE-bench Verified: Resolving Real-World GitHub Issues score?: Tools linked to SWE-bench Verified: Resolving Real-World GitHub Issues on Sophon include Mini SWE Agent Bench (Prime Community), Deepswe (Prime Intellect), Mini SWE Agent Plus, Opencode SWE - RL environments, datasets, and scaffolds that target this eval.
What license is SWE-bench Verified: Resolving Real-World GitHub Issues under?: SWE-bench Verified: Resolving Real-World GitHub Issues is available under mit.