Google (Alphabet Inc.)

Reading comprehension dataset that queries for complex, non-factoid information, and require difficult entailment-like inference to solve.

Evaluates how well language models can strictly follow detailed instructions, such as writing responses with specific word counts or including required keywords.

Tests AI models on a suite of 23 challenging BIG-Bench tasks that previously proved difficult even for advanced language models to solve.

Measures the ability of language models to generate short Python programs from simple natural-language descriptions, testing basic coding proficiency.