clbench

Tencent CL-bench long-context QA tasks: the model answers a long-context conversation and a strict, all-or-nothing LLM judge grades its final reply against the task's rubrics, returning an Overall Score of 0 or 1 that becomes the reward.

Taskset

• Source: tencent/CL-bench
• Size: 1899 tasks

Notes

• The judge grades all-or-nothing: every rubric must be satisfied for Overall Score 1; an empty completion or judge error scores 0.
• Prompts preserve the dataset's message structure, so they require a message-prompt harness (e.g. default). This also lets Verifiers transfer very large single-turn contexts without exceeding process argument limits.

Changelog

• 2026-08-31: Yield task records on demand so bounded evaluations construct only the requested prefix.
• 2026-08-30: Standardized the binary all-or-nothing grader on ReferenceJudge while retaining the official rubric prompt and judge diagnostics.
• 2026-08-27: Preserve single-turn prompts as messages so Verifiers can transfer long contexts without exceeding process argument limits.
• 2026-07-10: Initial v1 taskset.