
Leaderboard and traces · Blog post · Valen Labs
Machine learning tasks in computational biology, for coding agents. An agent reads the task statement, works
in /workspace, and writes a prediction script to /app/submission. Grading re-runs that
script twice on a sealed held-out split in a fresh, network-disabled sandbox and scores its
output. The hidden split and the scorer never enter the agent's sandbox.
Scores are rebased so that the task's reference method scores 0 and a perfect submission scores 100. A submission that fails any validity check scores 0.
Scope of this leaderboard
This environment ships three of the 25 FrontierBioML tasks, so its leaderboard ranks models on those three only, and the order differs from the full benchmark. The full leaderboard, 11 models across all 25 tasks with a trace for every run, is at frontierbioml.com, and the blog post describes the benchmark and the results.
Grader access
Scoring sends the submission to a hosted grader and needs a grader token in
FRONTIERBIOML_GRADER_TOKEN. Tokens are available on request from
Valen Labs. The held-out split is never distributed.
Tasks
| Task | Title | What the agent predicts |
|---|---|---|
t01 | Nanobody binder triage | Binding probability for every VHH-IL-6 pair, and a family-diverse candidate panel |
t09 | Docking pose scoring for antibody complexes | A score for every docking pose, so the top-scored pose in each panel is near-native |
t22 | Partner-selective KRAS variants | KRAS folding and partner binding, and variants that keep one partner and lose another |
Data sources and licences
Each task redistributes third-party data. The full notice, including the changes made, ships with
the task at environment/input/licenses/DATA-SOURCE.md.
| Task | Dataset | Source | Licence |
|---|---|---|---|
t01 | AVIDa-hIL6 | COGNANO — dataset | CC BY-NC 4.0 |
t09 | DeepRank-Ab | Zenodo 17911452 | CC BY 4.0 |
t22 | KRAS deep mutational scan | Weng, Faure, Escobedo and Lehner, Nature 626, 643-652 (2024) — doi | CC BY 4.0 |
t01 is licensed CC BY-NC 4.0: noncommercial use only. Running this task, and any use of its
data or results, must be noncommercial. t22 additionally redistributes six PDB entries from the
RCSB Protein Data Bank, unchanged; please cite those entries individually.
Changelog
- 0.1.7 — Describe the benchmark as machine learning tasks in computational biology; updated header image.
- 0.1.6 — README: FrontierBioML header image instead of the title and leaderboard graphic.
- 0.1.5 — README: updated leaderboard graphic.
- 0.1.4 — README: leaderboard graphic, links to the website and the blog post.
- 0.1.3 — the hosted grader moves to its new address (the
frontierbioml-graderapp). - 0.1.2 — README: scope of the leaderboard, and how to get a grader token.
- 0.1.1 — fix the taskset export:
frontierbioml/__init__.pyimported from the old package name. - 0.1.0 — first release:
t01,t09,t22, each with its data-source notice.