legal reasoning

Slug: legal-reasoning
Evals: 2
Tools: 7
Models: 5

Evals testing this capability

LegalBench

Stanford University

162 collaboratively curated legal-reasoning tasks across rule-recall, issue-spotting, application, and interpretation - the standard legal LLM benchmark.

ActiveLegal ReasoningFactual RecallLegal

Vals.ai Legal Evals

Vals AI

Vals.ai's proprietary suite of legal-domain benchmarks (contract review, hallucination tests, LegalBench Pro) used by law firms to procure LLMs.

ActiveLegal ReasoningHallucinationFactual RecallLegal

Tools lifting evals here

View all

Abercrombie RL Env (Community)

Teach a small model to classify trademarks on the Abercrombie distinctiveness spectrum

RL EnvLawTrademarkAbercrombieLegal

lifts 1 eval here

Legalbench RL Env (Community)

LegalBench environment for legal reasoning tasks

RL EnvLegalbenchReasoningLegal

lifts 1 eval here

Legalbench RL Env (Community)

Prime Intellect Verifiers environment for LegalBench legal reasoning tasks

RL Env

lifts 1 eval here

Legalbench RL Env (Community)

LegalBench environment for legal reasoning tasks

RL EnvLegalbenchReasoningLegal

lifts 1 eval here

Legalbench RL Env (Prime Community)

Prime Community

LegalBench environment for legal reasoning tasks

RL EnvLegalbenchReasoningLegal

lifts 1 eval here

Legalbench RL Env (Prime Intellect)

Prime Intellect

LegalBench environment for legal reasoning tasks

RL EnvLegalbenchReasoningLegal

lifts 1 eval here

Tomagpt RL Env (Community)

Legal reasoning and hearsay classifier

RL EnvLawLegalbenchHearsayLegal

lifts 1 eval here

Top models on this capability

by avg parsed score across evals here

legal reasoningBar chart with 5 bars. Highest value: GPT-4o-mini at 72.

5 models