DeepDive RLM

RLM (Recursive Language Model) environment for DeepDive - complex QA with Google search.

Overview

• Environment ID: deepdive-rlm
• Short description: Complex QA using RLM pattern with Google search tools (configurable placement on root, sub-LLMs, or both).
• Tags: qa, multiturn, search, tool-use, rlm

How It Works

This environment uses the Recursive Language Model (RLM) pattern:

1. Root Model: Writes Python code in a REPL environment to orchestrate the search process
2. Sub-LLMs: Called via llm_batch(prompts) function
3. Search tools (search_web, scan_page, open_lines): Available on the root LLM, inside the REPL, on sub-LLMs, or any combination (controlled by tools_on_root, tools_in_repl, tools_on_sub; default: sub-LLMs only)
4. Final Answer: Set via answer["content"] = "your answer" and answer["ready"] = True

This pattern is useful for complex queries that benefit from decomposition and recursive reasoning.

Datasets

• Primary dataset(s): DeepDive (arxiv, Huggingface)
• Split sizes: 2k train, 0.2k eval

Other datasets also work out of the box:

• RLinf/WideSeek-R1-train-data (search Q&A from WideSeek-R1)
• jmhb/PaperSearchQA (PubMed paper search from PaperSearchQA)
• OpenResearcher/OpenResearcher-Dataset — use dataset_subset="seed_42" (or seed_43 through seed_57) and dataset_split="train"

Setup and Install

uv run vf-install deepdive

You will also need an API key from Serper

Eval

Set all environment variables required for running the model and judge. For example, the judge defaults to Pinference's openai/gpt-4.1-mini, so you need to set the PRIME_API_KEY:

export PRIME_API_KEY=<your-key>
export SERPER_API_KEY=<your-serper-key>

Example evaluation:

prime eval run deepdive -m gpt-5-mini -n 5

Environment Arguments

Metrics

Changelog

• 0.2.13: Add a startup cache smoketest (write/read/delete round-trip) so misconfigured caches (wrong dir, no write permission, full disk, corrupt SQLite) raise a clear RuntimeError from configure_cache instead of silently turning every fetch into a cache-flavored error. Also shorten the TTL for cached fetch errors from cache_ttl_seconds (1 week) to a new error_cache_ttl_seconds (60s default) so transient failures don't pin a URL as broken; errors are no longer mirrored into the no-TTL mem cache.
• 0.2.12: Extend the judge prompt with a non-commit clause so refusal-style answers ("the answer cannot be determined", "I don't know", etc.) are scored as incorrect rather than getting credit.
• 0.2.11: Default judge requests now use Pinference (https://api.pinference.ai/api/v1) with PRIME_API_KEY and the Pinference-qualified openai/gpt-4.1-mini model name.
• 0.2.10: Add max_concurrent_search argument to make the parallel-query limit of search_web user-configurable (default unchanged at 10)
• 0.2.9: Replace tool_placement with tools_on_root, tools_in_repl, tools_on_sub flags for flexible tool placement. include_env_tips is temporarily a no-op (returns empty string with a warning) pending update for the new flags.
• 0.2.8: Add tool_placement argument to control whether search tools go to root, sub-LLMs, or both
• 0.2.7: Add missing dataset_* arguments to README and the new dataset_subset argument to the environment
• 0.2.6: align arg names with simplified RLMEnv (max_iterations → max_turns, sub_tool_max_turns → sub_llm_max_turns, sandbox params → sandbox_* prefix)
• 0.2.5: sandbox labels no longer force in the default label
• 0.2.4
  • Bump to verifiers>=v0.1.11.dev0 to support new types
• 0.2.3
  • Add prompt_in_context_file option to move the prompt into context.txt and leave the user prompt empty.
• 0.2.2
  • Validate sandbox_labels is a list of strings and always include deepdive-rlm.
  • Stop rollouts on Serper API failures and return 0 reward when they occur.