Allen Institute for AI (Ai2)

214 realistic, time-consuming web-research tasks ("how long is the train from X to Y on Tuesday?") that require live browsing and multi-page synthesis.

1,024 real, long, challenging user prompts mined from WildChat conversations and judged by GPT-4 with rubric-based pairwise scoring.

2025 successor to RewardBench - harder, multi-completion (not just chosen-vs-rejected), with refreshed prompts to address contamination.

2,985 prompt-chosen-rejected triples across chat, reasoning, safety, and code - a benchmark for evaluating reward models and LLM judges.

Grade-school science multiple-choice questions (Easy and Challenge sets) drawn from US standardized tests - an early language-understanding benchmark.

Allen AI's flagship open SFT mixture combining new persona-driven prompts with curated public data for post-training a frontier-quality instruct model.

1M real user-chatbot conversations collected by Allen AI / UW via a free GPT proxy - a window into how real users actually prompt LLMs.

Allen AI's open-source post-training codebase implementing SFT, DPO, and RLVR - the reference recipe behind Tülu 3 and OLMo-Instruct models.