How can a model improve its IFEval: Instruction-Following Evaluation score?

Tools linked to IFEval: Instruction-Following Evaluation on Sophon include Ifeval RL Env (Arcee AI), Allenai Ifeval RL Env (Dev Team), Ifeval RL Env (Community), Allenai Ifeval RL Env (Prime Intellect) - RL environments, datasets, and scaffolds that target this eval.

What license is IFEval: Instruction-Following Evaluation under?

IFEval: Instruction-Following Evaluation is available under mit.

IFEval: Instruction-Following Evaluation

Active

Evaluates how well language models can strictly follow detailed instructions, such as writing responses with specific word counts or including required keywords.

Open

Publisher: Google (Alphabet Inc.)
Domain: Reasoning
License: mit
Published: Oct 2024
Notable for: Benchmark for evaluating Reasoning.
Canonical: github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/ifeval

Cite

Notes

Only stored in your browser.

Attribution

README: github.com/UKGovernmentBEIS/inspect_evals/blob/main/src/inspect_evals/ifeval/eval.yamlMIT

Attribution policy →

Related tools

View all

Implementations, trainers, datasets and scaffolds linked to this eval.

Ifeval RL Env (Arcee AI)

Arcee AI

IFEval single-turn chat environment using RLVR-IFeval with JSON constraint rewards. Heavily inspired by and incorporates a lot of Allen AI's RLVR c...

ImplementationRL EnvIfevalConstraintsNone Reasoning

Allenai Ifeval RL Env (Dev Team)

Dev Team

IFEval single-turn environment using AllenAI RLVR-IFeval

ImplementationRL EnvIfevalConstraints

Ifeval RL Env (Community)

IFEval instruction following environment for Verifiers

ImplementationRL EnvInstruction FollowingIfevalConstraints

Allenai Ifeval RL Env (Prime Intellect)

Prime Intellect

IFEval single-turn environment using AllenAI RLVR-IFeval

ImplementationRL EnvIfevalConstraints

Ifeval RL Env (Prime Intellect)

Prime Intellect

IFEval evaluation environment

ImplementationRL EnvIfeval

Backdoor Ifeval RL Env (Community)

Blog-grounded Backdoor IFEval reward-hacking environment with hidden silver reward.

ImplementationRL EnvReward HackingBackdoorIfeval

Ifeval ALL RL Env (Prime)

Prime

Unified backdoor-ifeval env: difficulty, aggregation, no-v check, inoculation, group monitors

Trains towardRL EnvInstruction FollowingReward HackingBackdoor

Ifeval Groups RL Env (Prime)

Prime

Backdoor-ifeval env with group-level reward monitors for within-batch advantage variance

Trains towardRL EnvInstruction FollowingReward HackingBackdoor

Ifeval INOC RL Env (Prime)

Prime

Backdoor-ifeval env for inoculation experiments (pre-no-v version)

Trains towardRL EnvInstruction FollowingReward HackingBackdoor

Ifeval MINI RL Env (Community)

Reward hacking sprint calibration environment for hidden keyword gradients in instruction following.

Trains towardRL EnvReward HackingIfeval

FAQ

What is IFEval: Instruction-Following Evaluation?: Evaluates how well language models can strictly follow detailed instructions, such as writing responses with specific word counts or including required keywords.
How can a model improve its IFEval: Instruction-Following Evaluation score?: Tools linked to IFEval: Instruction-Following Evaluation on Sophon include Ifeval RL Env (Arcee AI), Allenai Ifeval RL Env (Dev Team), Ifeval RL Env (Community), Allenai Ifeval RL Env (Prime Intellect) - RL environments, datasets, and scaffolds that target this eval.
What license is IFEval: Instruction-Following Evaluation under?: IFEval: Instruction-Following Evaluation is available under mit.