Swebenchpro
Frontier
SWE-bench Pro: A multi-language software engineering benchmark with 731 instances covering Python, JavaScript/TypeScript, and Go. Evaluates AI systems' ability to resolve real-world bugs and implement features across diverse production codebases. Original benchmark: https://github.com/scaleapi/SWE-bench_Pro-os. Adapter details: https://github.com/laude-institute/harbor/tree/main/adapters/swebenchpro
- Domain
- agent-eval
- Published
- Nov 2025
Cite
Notes
Only stored in your browser.
Top score 80.3% by Claude Fable 5 - 5 models reporting (4 frontier)
Score history
4Top models
5
FAQ
- What is Swebenchpro?
- SWE-bench Pro: A multi-language software engineering benchmark with 731 instances covering Python, JavaScript/TypeScript, and Go. Evaluates AI systems' ability to resolve real-world bugs and implement features across diverse production codebases. Original benchmark: https://github.com/scaleapi/SWE-bench_Pro-os. Adapter details: https://github.com/laude-institute/harbor/tree/main/adapters/swebenchpro
- What is the current top score on Swebenchpro?
- The top reported score is 80.3% by Claude Fable 5, across 5 models reporting (4 from frontier labs).