What is the current top score on PubMedQA: A Dataset for Biomedical Research Question Answering?

The top reported score is 77.5% by Claude Opus 4.5, across 2 models reporting (2 from frontier labs).

How can a model improve its PubMedQA: A Dataset for Biomedical Research Question Answering score?

Tools linked to PubMedQA: A Dataset for Biomedical Research Question Answering on Sophon include Pubmedqa RL Env (Community), Openmed Pubmedqa RL Env (Community), Pubmedqa RL Env (Medarc), Openmed Pubmedqa RL Env (Community) - RL environments, datasets, and scaffolds that target this eval.

PubMedQA: A Dataset for Biomedical Research Question Answering

Active

Biomedical question answering (QA) dataset collected from PubMed abstracts.

Open

Publisher: University of Pittsburgh
Domain: Knowledge
License: mit
Published: May 2026
Notable for: Benchmark for evaluating Knowledge.
Canonical: github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/pubmedqa

Cite

Notes

Only stored in your browser.

Attribution

README: github.com/UKGovernmentBEIS/inspect_evals/blob/main/src/inspect_evals/pubmedqa/eval.yamlMIT
Leaderboard scores: prime-hub

Attribution policy →

Top score 77.5% by Claude Opus 4.5 - 2 models reporting (2 frontier)

Score history

Top models

PubMedQA: A Dataset for Biomedical Research Question AnsweringBar chart with 2 bars. Highest value: Claude Opus 4.5 at 77.5.

2 models

Related tools

View all

Implementations, trainers, datasets and scaffolds linked to this eval.

Pubmedqa RL Env (Community)

Single-turn medical MCQ

ImplementationRL EnvMedicalClinical

Openmed Pubmedqa RL Env (Community)

Biomedical research QA environment - yes/no/maybe classification on PubMed abstracts

ImplementationRL EnvScientific ReasoningClassificationOpenmed

Pubmedqa RL Env (Medarc)

Medarc

Single-turn medical MCQ

ImplementationRL EnvMedicalClinical

Openmed Pubmedqa RL Env (Community)

Single-turn biomedical QA environment for PubMedQA with CoT and boxed yes/no/maybe decision

ImplementationRL EnvBiomedicalOpenmedPubmedqa

Search L 2 RL Env (Community)

Bio-Literature Search L2: Multi-abstract comparison - section existence, MeSH overlap, decision matching, paragraph counting

Trains towardRL EnvTool UseBiomedicalBiology

Search L 3 RL Env (Community)

Bio-Literature Search L3: Open-ended scientific synthesis with LLM judge (GPT-4.1-mini)

Trains towardRL EnvTool UseBiomedicalBiology

Openmed Bioasq RL Env (Community)

Biomedical question answering environment using PubMedQA dataset - essential for evidence-based medicine

Trains towardRL EnvOpenmedPubmedqaBioasq

Pubmedqa Yesno RL Env (Community)

Biomedical yes/no/maybe QA environment using PubMedQA

Trains towardRL EnvBiomedicalClassificationMedical

FAQ

What is PubMedQA: A Dataset for Biomedical Research Question Answering?: Biomedical question answering (QA) dataset collected from PubMed abstracts.
What is the current top score on PubMedQA: A Dataset for Biomedical Research Question Answering?: The top reported score is 77.5% by Claude Opus 4.5, across 2 models reporting (2 from frontier labs).
How can a model improve its PubMedQA: A Dataset for Biomedical Research Question Answering score?: Tools linked to PubMedQA: A Dataset for Biomedical Research Question Answering on Sophon include Pubmedqa RL Env (Community), Openmed Pubmedqa RL Env (Community), Pubmedqa RL Env (Medarc), Openmed Pubmedqa RL Env (Community) - RL environments, datasets, and scaffolds that target this eval.
What license is PubMedQA: A Dataset for Biomedical Research Question Answering under?: PubMedQA: A Dataset for Biomedical Research Question Answering is available under mit.