CvxpyRust

A long-horizon OpenReward environment: the agent re-implements the core of cvxpy (a convex-optimization modeling library) from scratch in pure-std Rust as cvxsolve — DCP curvature/sign analysis, the is_dcp verdict, and a from-scratch convex solver — graded server-side against a cvxpy-generated corpus. Published as GeneralReasoning/CvxpyRust.

See DESIGN.md for the full design and build/PROTOCOL.md for the artifact invocation + grading contract.

Reward

Per-build delta of the build's score: the current build's score minus the previous build's, so regressions are negative. Reward tracks the current artifact, so the trajectory total telescopes to the score of the build the agent finishes with. finished=True only when a single build passes every case at once.

Each tier is worth its case count (857 in all). An answer that ignores the input earns 0:

• classification questions (curvature and sign in tier1, half a case each; is_dcp in tier2) earn chance-corrected balanced accuracy, cases × (BA − 1/K)/(1 − 1/K) (Youden's J for is_dcp), unclipped. A missing or unusable answer counts as unknown / false, so a crash scores 0;
• solve cases a constant answer passes (infeasible/unbounded, and v* = 0) are credited in proportion to the pass rate on the tier's other cases.

run_tests

• timeout_s is clamped to 60–3600s; the producer's sandbox.run gets a deadline above it and the build its own 900s one (the SDK default is 300s).
• Every producer process runs under a 1.5 GiB address-space limit (ulimit -v), so a case that allocates without bound fails on its own. If the sandbox still runs out of memory (e.g. huge files under /tmp), run_tests returns "ran out of memory" feedback (reward 0, baseline untouched, no grade charged) and stops the leftover runner/jail processes and scratch files instead of raising.
• The graded corpus is NOT on the sandbox: the env-server image holds it and each grade stages only the questions into a random /tmp dir, deleted afterwards.
• Calls are serialized per session, so a concurrent build can't swap the binary under a running grade.
• The tool output is bounded (metadata included). The full build log and the full report go to /tmp/run_tests/{build.log,report.txt} in the sandbox (previous build: .prev), written in chunks and best-effort (a sabotaged /tmp can't error the rollout). Per-case pass/fail is deliberately NOT written: the answers are small categorical labels (see DESIGN.md).

Layout

Develop / test

# No-Docker invariant tests:
uv run python -m pytest golden_tests.py -v

# Local sandbox-image build (iterate / reference-solution check). On push to main
# .github/workflows/build-sandbox.yml builds + pushes :latest + :sha-<gitsha> to
# GHCR; then repin SANDBOX_IMAGE to the new :sha-<gitsha> (NOT :latest — the GHCR
# mirror serves :latest stale). Build + push by hand only as a fallback:
docker build --platform linux/amd64 -f sandbox.Dockerfile -t ghcr.io/generalreasoning/cvxpyrust:latest .
docker push ghcr.io/generalreasoning/cvxpyrust:latest

# Local server smoke test:
uv run python server.py            # in one shell
uv run python test_agent.py        # in another

Corpus regeneration (author-time only)

# needs cvxpy; produces the committed build/corpus/ (do not run in either image)
python build/generate_corpus.py --out build/corpus
# discrimination check against a built reference cvxsolve:
python build/local_grade.py --binary reference_solution/target/release/cvxsolve
python build/local_grade.py --baseline   # the ~25% always-constant floor to beat