0

Multiswe

Fresh

PrimeIntellect Multi-SWE taskset.

Type
RL Env
Publisher
Prime Intellect
Tags
V1
Runtime
multi-turn
License
unknown
Size
v0.1.4
Published
Aug 2026
Updated
Sep 2026
Canonical
app.primeintellect.ai/dashboard/environments/primeintellect/multiswe

Cite

Notes

Only stored in your browser.

Attribution

Attribution policy →

multiswe

Multi-SWE-bench issue-resolving tasks across many languages, each backed by a prebuilt sandbox image. Scored by extracting a source-only fix patch, running the task's fix-run.sh harness, and validating the resulting report against the expected fail/pass tests via upstream multi_swe_bench report generation.

Taskset

Changelog

  • 2026-09-04: Add a solver system prompt (adapted from FrontierCode 1.1's fair-internet-use prompt) that allows documentation lookups but forbids retrieving the task's upstream fix online or from non-current-branch git history; complements #795, which restored network access.
  • 2026-09-03: Restore default solver network access by removing the network_allow=[] override introduced in #780; training rollouts need outbound network.
  • 2026-08-31: Yield task records on demand so bounded evaluations construct only the requested prefix.
  • 2026-08-27: Install multi-swe-bench==1.1.2 from its PyPI source archive because the published wheel contains conflicting duplicate entries and current uv rejects it; restore the lowercase Qiskit adapter omitted by that source archive.
  • 2026-07-17: patch_capture now imports from verifiers.v1 (capture_patch/resolve_head upstreamed in verifiers#2054); the copied module is removed. Requires verifiers>=0.2.2.dev5.
  • 2026-07-17: Task images now use the raw source refs (docker_image as-is, or the upstream Docker Hub mswebench/<org>_m_<repo>:pr-<number> ref) instead of rewriting to prime/primeintellect/. The Prime platform serves these natively (org-less platform copies under the source names; missing ones auto-import from Docker Hub on first use), so no registry mapping is needed.
  • 2026-07-16: finalize persists the agent's final patch to trace.info["patch"] (truncated at 2 MB with patch_truncated; failures record patch_error instead of failing the rollout) via the copied patch_capture helper.
  • 2026-07-10: Ported to the task-centric verifiers API: rewards and lifecycle hooks live on the Task (a TaskData row + behavior split), and task-facing config knobs (judges, tool/user placement, scoring parameters) moved from --env.taskset.* to --env.taskset.task.*. Requires verifiers>=0.2.0 and Python >=3.11.
  • 2026-07-08: Task images now resolve from the public Prime platform registry (prime/primeintellect/<name>:<tag>, mswebench/ namespace stripped) instead of the private GCP Artifact Registry mirror; any Prime user can pull them.
  • 2026-07-06: Default dataset renamed to PrimeIntellect/Multi-SWE-RL-Verified (formerly Multi-SWE-RL); the old name redirects.
  • 2026-07-01: Added filter_fn, applied directly with datasets.Dataset.filter to raw HF rows before task construction.
  • 2026-07-01: Default dataset is now PrimeIntellect/Multi-SWE-RL. Known PrimeIntellect datasets are typed on MultiSWEConfig.dataset_name; known datasets get split validation for their available splits.
  • 2026-06-30: Setup now moves top-level /home grading scripts and /home/test.patch through a host-side archive and removes them from the sandbox. Scoring restores the archived artifacts before running /home/fix-run.sh. This intentionally diverges from the original Multi-SWE images, where those artifacts remain visible, because the v1 live sandbox would otherwise let agents read or edit grading material.