Humaneval (Pmahdavi)

Fresh

HumanEval code generation evaluation environment

Cite

Notes

Only stored in your browser.

Attribution

README: api.primeintellect.ai/api/v1/environmentshub/pmahdavi/humaneval/@0.1.1/inspect

Evals this tool implements

Same problem set, this tool's harness. Run it to score a model on the test.

OpenAI

164 hand-written Python programming problems with unit tests, the original LLM code-generation benchmark from OpenAI's Codex paper.

OpenAI

Assesses how accurately language models can write correct Python functions based solely on natural-language instructions provided as docstrings.