marketmaker

OpenReward environment where the agent writes a market-making strategy as Python code, submits it, and the env runs it inside a deterministic CLOB simulator against historical L2 BTC tick data sliced into 15-minute "markets." The agent's score is the strategy's aggregate PnL across the markets it commits to trading.

Documentation

• MM_ONE_PAGER.md — design summary for reviewers (CLOB primer, tool surface, strategy API, key mechanics, grading, data, references).
• PLAN.md — full design spec.

Layout

mm/                  CLOB simulator + Strategy harness + parquet loader
examples/            Reference strategies (constant_spread, imbalance_quoter,
                     avellaneda_stoikov, self_timer)
infra/               Standalone Crypto-Lake → 15-min anonymized parquet pipeline
                     (NOT shipped at runtime)
tests/               In-process tests for the env's burn-rule state machine
scripts/             Misc tooling (md_to_html.py, …)
env.py / specs.py / tasks.py / server.py
                     OpenReward Environment subclass + tool handlers
try_env.py           Local in-process driver — exercises the full tool surface
                     against one task without standing up the HTTP server
Dockerfile           Production image (no data baked in; reads /orwd_data
                     mounted by OpenReward at runtime)

Quickstart (local dev)

Python 3.11+ required.

# Set up venv + deps
py -3.11 -m venv .venv
.venv/Scripts/python -m pip install -r requirements.txt

# Build a small dev dataset (1 day of free Crypto-Lake BTC ≈ 187 MB, 96 markets)
.venv/Scripts/python infra/build_dataset.py --use-sample \
    --start 2022-10-01 --end 2022-10-02 \
    --slice-min 15 --out data_dev/markets_test

# Run a baseline strategy on one real market with verbose output
.venv/Scripts/python -m mm.run \
    --script examples/constant_spread.py \
    --market-dir data_dev/markets_test/markets/M_00045 \
    --position-cap 5 -v

# Drive the OpenReward env tool surface end-to-end (in-process, no HTTP)
.venv/Scripts/python try_env.py

# Run the burn-rule test suite
.venv/Scripts/python tests/test_env_rules.py

Reward

Sum of per-market log-wealth changes across the markets the agent committed to trading:

reward = Σ_i log( (initial_cash + pnl_i) / initial_cash )

Researched markets contribute zero. Untouched markets contribute zero. Bigger reward is better; conversion to a portfolio percent-change is pct ≈ (exp(reward) - 1) × 100%. Sharpe / max-drawdown / longest-losing-streak are computed and surfaced alongside as informational metrics. See MM_ONE_PAGER.md for the full grader spec.

Safety

• Production: agent-submitted strategies execute in an OpenReward sandbox container, isolated from the env. (V1 of this repo runs them in-process for local testing — clearly TODO'd in env.py and PLAN.md §3.)
• The env itself has no network access and no filesystem writes outside the working dir.
• Reward is deterministic, computed from a closed-form non-LLM grader.

License

MIT.

Citation

@dataset{GeneralReasoningMarketMaker2026,
  author    = {General Reasoning},
  title     = {marketmaker: a script-submission market-making environment},
  year      = {2026},
  publisher = {OpenReward},
  url       = {https://openreward.ai/GeneralReasoning/marketmaker}
}