Prime Community

Prime Community is a team.

Type: team

Cite

Notes

Only stored in your browser.

Evals

0

Tools

44

Models

0

Papers

0

Boards

0

People

0

Tools

44

NYT Connections RL Env (Prime Community)

Word puzzle game where players find groups of 4 words sharing a common theme

RL EnvWord PuzzleReasoningWord PuzzlesGames

WEB PY RL Env (Prime Community)

Humanity's Last Examination (HLE) benchmark environment for Prime Community Environments

RL EnvHleMulti ModalTool UseMultimodal

Polars ENV RL Env (Prime Community)

Polars DataFrame manipulation environment for training and evaluation

RL EnvPolarsDataframeData Manipulation

Training Puzzles RL Env (Prime Community)

LLM Training Puzzles by Sasha Rush

Hanabi RL Env (Prime Community)

Hanabi game

RL EnvMulti Agent

Stepfun Prover RL Env (Prime Community)

A multi-turn RL environment for formal theorem proving in Lean 4, where models alternate between reasoning, sketching proof code, and receiving ver...

RL EnvMathTheorem ProvingLean4

Vision SR 1 RL Env (Prime Community)

Vision-SR1 environment (train+eval) using original graders

RL EnvVision Sr1Multi PassVlmMultimodal

Minif2f RL Env (Prime Community)

Verifiers port of the minif2f benchmark

RL EnvTheorem ProvingFormal VerificationMath

Skyrl SQL RL Env (Prime Community)

Multi-turn Text-to-SQL environment with interactive database feedback following SkyRL-SQL methodology

RL EnvSQLText to SQLDatabaseCode

Agent Bench RL Env (Prime Community)

Benchmarking model performance on SWE Bench in the Mini SWE Agent harness.

RL EnvTool UseAgent

Puzzles Modal RL Env (Prime Community)

GPU puzzles environment by Sasha Rush using modal sandboxes

RL EnvProgrammingCode

FH Aviary RL Env (Prime Community)

Future House Aviary wrapper for verifiers - Scientific reasoning environments with tools

RL EnvAviaryScientific ReasoningToolsScience

Dabstep RL Env (Prime Community)

Data Agent Benchmark for Multi-step Reasoning benchmark

RL EnvData AnalysisOpenai CompatibleCodingCode

ART E RL Env (Prime Community)

ART-E: a tool-using email research RL environment for Verifiers

RL EnvEmailResearchTool Use

Bigbench BBH RL Env (Prime Community)

Big Bench + BBH implementation

RL EnvBigbenchBbhNLP

Codebase Search RL Env (Prime Community)

Codebase search environment for Triton GPU programming library - tests agent's ability to navigate and answer questions about the Triton codebase u...

RL EnvCodebase SearchTool Use

Sycophancy EVAL RL Env (Prime Community)

Evaluates sycophantic behavior in LLMs across four tasks from Sharma et al. (ICLR 2024).

RL EnvSycophancyBiasLanguage Models

Agentharm RL Env (Prime Community)

AgentHarm environment to evaluate agentic reasoning and safety

RL EnvSafetyAgentTool UseSecurity

Spiral Bench RL Env (Prime Community)

Multi-turn text-based environment for evaluating agents on the Spiral-Bench dataset.

RL EnvLong HorizonSafetyRiskSecurity

Fruit BOX RL Env (Prime Community)

text-based muli-turn fruit box game environment

RL EnvStrategyGrid BasedGames

Backend Bench RL Env (Prime Community)

BackendBench environment for LLM kernel benchmarking

Androidworld RL Env (Prime Community)

AndroidWorld benchmark for evaluating autonomous agents on real Android apps with 116 tasks across 20 apps

RL EnvMobileAndroidTool UseMultimodal

Legalbench RL Env (Prime Community)

LegalBench environment for legal reasoning tasks

RL EnvLegalbenchLegalReasoning

Agent DOJO RL Env (Prime Community)

Benchmark for agent robustness against prompt injection attacks in tool-use scenarios

RL EnvSecurityPrompt InjectionTool Use

Seeclick RL Env (Prime Community)

Test model's ability to correctly click on target UI

RL EnvMultimodal

Bixbench RL Env (Prime Community)

BixBench scientific reasoning evaluation environment

RL EnvScientific ReasoningOpen AnswerScience

Agent Bench RL Env (Prime Community)

A realistic virtual EHR environment to benchmark medical LLM agents on clinical tasks.

RL EnvMedicalEhrClinical

Mastermind RL Env (Prime Community)

Mastermind multi-turn game environment for Verifiers

RL EnvGameReasoningGames

Clockbench RL Env (Prime Community)

ClockBench: multimodal clock reading and reasoning benchmark implemented for verifiers.

RL EnvClockbenchMultimodalVision

Aider Polyglot RL Env (Prime Community)

Multi-turn environment for testing coding abilities across multiple programming languages using Exercism exercises

RL EnvCodingPolyglotCode

Aidanbench RL Env (Prime Community)

AidanBench multi-turn environment for Verifiers

RL EnvAidanbenchNovelty

Gutenberg ENV RL Env (Prime Community)

Agentic RAG over Sherlock Holmes short stories for literary Q&A

RL EnvGutenbergAgentic SearchRAG

MCP Universe RL Env (Prime Community)

MCP Universe environment for evaluating LLMs in wide range of tasks with MCP server

RL EnvMcpToolsTool Calling

Transformerpuzzles RL Env (Prime Community)

TransformerPuzzles by Sasha Rush

RL EnvSandboxes

Bench ENV RL Env (Prime Community)

τ-bench: Tool-Agent-User benchmark for conversational agents in customer service domains with user simulation

RL EnvTau BenchConversationTool Use

TEXT Quests RL Env (Prime Community)

Classic Infocom interactive fiction games (Zork, Enchanter, etc.) for evaluating LLM reasoning, planning, and world modeling

RL EnvText GamesReasoningPlanning

MMLU RL Env (Prime Community)

MMLU evaluator for multi-subject multiple-choice reasoning.

RL EnvGeneral KnowledgeNLP

Negative Style RL Env (Prime Community)

ENV for self-grading for LLM Writer Style.

WIKI RACE RL Env (Prime Community)

Environment for the game Wiki Race

RL EnvGameNavigationGames

Scicode RL Env (Prime Community)

SciCode evaluation environment

RL EnvCodePythonStem

Github MCP RL Env (Prime Community)

GitHub MCP environment

RL EnvMcpGithub

AGI TOOL RL Env (Prime Community)

ARC-AGI 1 + 2 with tool calling (Abstract and Reasoning Corpus)

RL EnvArc AGITool UseReasoningGames

Balrog Bench RL Env (Prime Community)

BALROG benchmark integration for verifiers: unified RL evaluation across game environments.

Browsecomp PLUS RL Env (Prime Community)

Verifiers environment for BrowseComp-Plus Deep-Research Agent Benchmark. Controlled agent/retriever evaluation on the fixed human-verified corpus.

RL EnvSearch AgentDeep ResearchRetriever