How can a model improve its Measuring Massive Multitask Language Understanding score?

Tools linked to Measuring Massive Multitask Language Understanding on Sophon include Openmed Medknowledge RL Env (Community), MMLU RL Env (Prime Community), MMLU RL Env (Community) - RL environments, datasets, and scaffolds that target this eval.

What license is Measuring Massive Multitask Language Understanding under?

Measuring Massive Multitask Language Understanding is available under mit.

Measuring Massive Multitask Language Understanding

Active

Dataset Card for MMLU

Open

Publisher: University of California, Berkeley
License: mit
Published: May 2026
Canonical: huggingface.co/datasets/cais/mmlu

Cite

Notes

Only stored in your browser.

Sample tasks

from the eval dataset

abstract_algebra

Find the degree for the given field extension Q(sqrt(2), sqrt(3), sqrt(18)) over Q.

Show 4 more examples

abstract_algebra

Let p = (1, 2, 5, 4)(2, 3) in S_5 . Find the index of <p> in S_5.

A8
B2
C24
D120

abstract_algebra

Find all zeros in the indicated finite field of the given polynomial with coefficients in that field. x^5 + 3x^3 + x^2 + 2x in Z_5

A0
B1
C0,1
D0,4

abstract_algebra

Statement 1 | A factor group of a non-Abelian group is non-Abelian. Statement 2 | If K is a normal subgroup of H and H is a normal subgroup of G, then K is a normal subgroup of G.

ATrue, True
BFalse, False
CTrue, False
DFalse, True

abstract_algebra

Find the product of the given polynomials in the given polynomial ring. f(x) = 4x - 5, g(x) = 2x^2 - 4x + 2 in Z_8[x].

A2x^2 + 5
B6x^2 + 4x + 6
C0
Dx^2 + 1

[
  {
    "question": "Find the degree for the given field extension Q(sqrt(2), sqrt(3), sqrt(18)) over Q.",
    "subject": "abstract_algebra",
    "choices": [
      "0",
      "4",
      "2",
      "6"
    ],
    "answer": 1
  },
  {
    "question": "Let p = (1, 2, 5, 4)(2, 3) in S_5 . Find the index of <p> in S_5.",
    "subject": "abstract_algebra",
    "choices": [
      "8",
      "2",
      "24",
      "120"
    ],
    "answer": 2
  },
  {
    "question": "Find all zeros in the indicated finite field of the given polynomial with coefficients in that field. x^5 + 3x^3 + x^2 + 2x in Z_5",
    "subject": "abstract_algebra",
    "choices": [
      "0",
      "1",
      "0,1",
      "0,4"
    ],
    "answer": 3
  },
  {
    "question": "Statement 1 | A factor group of a non-Abelian group is non-Abelian. Statement 2 | If K is a normal subgroup of H and H is a normal subgroup of G, then K is a normal subgroup of G.",
    "subject": "abstract_algebra",
    "choices": [
      "True, True",
      "False, False",
      "True, False",
      "False, True"
    ],
    "answer": 1
  },
  {
    "question": "Find the product of the given polynomials in the given polynomial ring. f(x) = 4x - 5, g(x) = 2x^2 - 4x + 2 in Z_8[x].",
    "subject": "abstract_algebra",
    "choices": [
      "2x^2 + 5",
      "6x^2 + 4x + 6",
      "0",
      "x^2 + 1"
    ],
    "answer": 1
  }
]

Related tools

View all

Implementations, trainers, datasets and scaffolds linked to this eval.

Openmed Medknowledge RL Env (Community)

Comprehensive medical knowledge MCQ environment using MMLU medical subsets.

Trains towardRL Env

MMLU RL Env (Prime Community)

Prime Community

MMLU evaluator for multi-subject multiple-choice reasoning.

Trains towardRL EnvNLPGeneral Knowledge

MMLU RL Env (Community)

MMLU evaluator for multi-subject multiple-choice reasoning.

Trains towardRL EnvNLPGeneral Knowledge

FAQ

What is Measuring Massive Multitask Language Understanding?: Dataset Card for MMLU
How can a model improve its Measuring Massive Multitask Language Understanding score?: Tools linked to Measuring Massive Multitask Language Understanding on Sophon include Openmed Medknowledge RL Env (Community), MMLU RL Env (Prime Community), MMLU RL Env (Community) - RL environments, datasets, and scaffolds that target this eval.
What license is Measuring Massive Multitask Language Understanding under?: Measuring Massive Multitask Language Understanding is available under mit.