python-coding-base

HumanEval Python code generation environment (fork of mohamed313/python-coding-eval for a code-generation RL experiment).

The model completes a Python function; the completion is run against the HumanEval unit tests in an isolated subprocess with a timeout and import-safety checks.

Variant

• python-coding-base — original binary reward: 1.0 if all unit tests pass, else 0.0.
• python-coding-partial — modified reward (small first change): 1.0 if all tests pass, 0.3 if the code defines a callable entry point but fails some tests, else 0.0. Denser reward signal toward writing structurally valid code.

Usage

from verifiers import load_environment
env = load_environment("python-coding-base", num_examples=100, offset=0)

Notes

• Dataset: openai/openai_humaneval (split test, 164 problems).
• num_examples and offset select a slice of the 164 problems, so training and held-out eval can consume disjoint slices.
• Source: https://github.com/PrimeIntellect-ai/prime