0

Swelego

Fresh

SWE-Lego real-data issue-resolving tasks (agentic; sandbox pytest reward).

Type
RL Env
Publisher
Prime Intellect
Tags
V1
Runtime
multi-turn
License
unknown
Size
v0.1.4
Published
Aug 2026
Updated
Sep 2026
Canonical
app.primeintellect.ai/dashboard/environments/primeintellect/swelego

Cite

Notes

Only stored in your browser.

Attribution

Attribution policy →

swelego

SWE-Lego real-data issue-resolving tasks drawn from real GitHub issues, each with a public image carrying the repo. Scored 1.0 iff every FAIL_TO_PASS and PASS_TO_PASS test id passes after the held-out test patch is re-applied and the task's canonical test command is run.

Taskset

Changelog

  • 2026-09-04: Add a solver system prompt (adapted from FrontierCode 1.1's fair-internet-use prompt) that allows documentation lookups but forbids retrieving the task's upstream fix online or from non-current-branch git history; complements #795, which restored network access.
  • 2026-09-03: Restore default solver network access by removing the network_allow=[] override introduced in #780; training rollouts need outbound network.
  • 2026-08-31: Yield task records on demand so bounded evaluations construct only the requested prefix.
  • 2026-07-17: patch_capture now imports from verifiers.v1 (capture_patch/resolve_head upstreamed in verifiers#2054); the copied module is removed. Requires verifiers>=0.2.2.dev5.
  • 2026-07-16: finalize persists the agent's final patch to trace.info["patch"] (truncated at 2 MB with patch_truncated; failures record patch_error instead of failing the rollout) via the copied patch_capture helper.
  • 2026-07-10: Ported to the task-centric verifiers API: the solved reward and the setup hook now live on SWELegoTask (a SWELegoData row + behavior split), and the taskset exposes load instead of load_tasks. Load-time config (dataset_name, split, filter_fn) stays on --env.taskset.*. Requires verifiers>=0.2.0 and Python >=3.11.
  • 2026-07-08: Default dataset is now the gold-patch-validated PrimeIntellect/SWE-Lego-Real-Data-Verified (4,323 rows); the fixed-fork base SWE-Lego-Real-Data stays available as a typed override.
  • 2026-07-06: PrimeIntellect/SWE-Lego-Real-Data-Validated renamed to PrimeIntellect/SWE-Lego-Real-Data-Verified in the typed dataset options; the old name redirects.
  • 2026-07-01: Added filter_fn, applied directly with datasets.Dataset.filter to raw HF rows before task construction.
  • 2026-06-30: Setup no longer applies test_patch. Scoring applies the patch after resetting touched files from base_commit when available, matching the original SWE-Lego/SWE-bench eval pattern rather than the verifiers composable port's setup-time application.