0

Terminal-Bench

Active

Suite of real-world command-line tasks executed inside sandboxed Docker containers to measure agentic shell competence.

Open
Publisher
Laude Institute
Capabilities
Tool CallingPlanningCode EditingDebugging
Domain
agentic
Format
Custom
Size
80 tasks
License
Apache-2.0
Published
Jan 2025
Updates
Monthly
Notable for
The reference leaderboard for terminal / DevOps / sysadmin agent capability, complementary to SWE-bench's IDE-style coding focus.
Canonical
tbench.ai
Official leaderboard
tbench.ai/leaderboard
Also on
github.com/laude-institute/terminal-bench

Cite

Notes

Only stored in your browser.

Attribution

Leaderboard scores
prime-hub
Attribution policy →

Top score 26.7% by DeepSeek V3.1 - 1 model reporting (1 frontier)

Top models

1
Terminal-BenchBar chart with 1 bar. Highest value: DeepSeek V3.1 at 26.7.
1 model

Where it's ranked

1

Official leaderboard

tbench.ai

Single benchmark
monthly

Related tools

6
View all

Implementations, trainers, datasets and scaffolds linked to this eval.

Terminal Bench RL Env (Community)

Terminal-Bench wrapper environment for verifiers.

ImplementationRL Env

Terminalbench RL Env (Community)

Terminal-Bench wrapper environment for verifiers.

ImplementationRL EnvTerminalbenchTerminal

OpenEnv Terminus (HF Sandbox terminal)

Hugging Face

Single-tool (tmux session) terminal coding environment in the OpenEnv standard, backed by Hugging Face Sandbox containers - the OpenEnv port of the Terminus-2 design.

Trains towardRL EnvCode EditingPlanningTool Calling

Terminal-Bench (Verifiers wrapper)

Harbor

Terminal-Bench tasks wrapped as a Verifiers RL environment - model drives a tmux shell to complete realistic end-to-end terminal jobs (compiling, sysadmin, data science).

Trains towardRL EnvCode EditingPlanningTool Calling

Harbor RL Env (Prime Intellect)

Prime Intellect

Harbor (terminal-bench-style) tasks via ComposableEnv.

Trains towardRL EnvTerminal Bench

Bench 2 RL Env (Prime Intellect)

Prime Intellect

Terminal-Bench 2.1 Harbor taskset with Terminus2 as the default harness

Trains towardRL EnvCli AgentTerminal Bench

Papers

3

Terminal-Bench: A Benchmark for Real-World Terminal-Based Agents

blog · 2025

Laude Institute + Stanford benchmark of containerized terminal tasks for evaluating shell-based coding agents - released as a public leaderboard + paper.

introduces

Introducing Terminal-Bench

blog · 2025

Laude Institute + Stanford launch blog for Terminal-Bench, a benchmark of real shell-based tasks executed inside sandboxed containers.

Terminal-Bench: A Benchmark for Real-World Terminal-Based Agents

blog · 2025

Laude Institute + Stanford benchmark of containerized terminal tasks for evaluating shell-based coding agents - released as a public leaderboard + paper.

Contributors

2
MMike MerrillAAlex Shaw

FAQ

What is Terminal-Bench?
Suite of real-world command-line tasks executed inside sandboxed Docker containers to measure agentic shell competence.
What capabilities does Terminal-Bench test?
Terminal-Bench evaluates tool calling, planning, code editing, debugging.
What is the current top score on Terminal-Bench?
The top reported score is 26.7% by DeepSeek V3.1, across 1 model reporting (1 from frontier labs).
How can a model improve its Terminal-Bench score?
Tools linked to Terminal-Bench on Sophon include Terminal Bench RL Env (Community), Terminalbench RL Env (Community), OpenEnv Terminus (HF Sandbox terminal), Terminal-Bench (Verifiers wrapper) - RL environments, datasets, and scaffolds that target this eval.
What license is Terminal-Bench under?
Terminal-Bench is available under Apache-2.0.