retrieval

214 realistic, time-consuming web-research tasks ("how long is the train from X to Y on Tuesday?") that require live browsing and multi-page synthesis.

1,266 hard fact-finding questions on the open web requiring persistent browsing and reasoning over scattered, obscure sources.

Classic long-context stress test that hides a short fact ("the needle") inside a long document and asks the model to retrieve it.

BrowserEnv[DOM] Environment on Browsecomp

Tool-use environment for the model to browse the web and locate hard-to-find information; scored using an LLM-as-judge rubric

Tool-use environment for the model to browse the web and locate hard-to-find information; scored using an LLM-as-judge rubric

BrowseComp evaluation environment

ServiceNow's unified Gym-style framework for web agents - wraps WebArena, MiniWoB, VisualWebArena, WorkArena, AssistantBench, WebLINX, and more under one Playwright-backed interface.

Needle-in-haystack - locate a target sentence in a long document.

browsecomp with deepdive tools

BrowseComp with DeepDive tools using RLM

Multi-turn open-web research QA environment where the model uses a Serper search tool to answer hard, hop-heavy questions of the BrowseComp / SimpleQA style.

Needle-in-haystack environment using RLM with Python REPL