Chess Environment

A chess reinforcement learning environment for OpenEnv, powered by the moonfish chess engine.

Features

Full chess rules via python-chess library
Configurable opponent: moonfish engine, random moves, or self-play
Position evaluation: Uses moonfish's PSQT-based evaluation
Standard OpenEnv interface: reset(), step(), state

Quick Start

Using Docker

# Build the image
docker build -t chess-env:latest -f envs/chess_env/server/Dockerfile .

# Run the server
docker run -p 8000:8000 chess-env:latest

Using the Client

The client is async by default:

import asyncio
from chess_env import ChessEnv, ChessAction

async def main():
    async with ChessEnv(base_url="http://localhost:8000") as env:
        # Reset for a new game
        result = await env.reset()
        print(f"Starting position: {result.observation.fen}")
        print(f"Legal moves: {result.observation.legal_moves}")

        # Make a move
        result = await env.step(ChessAction(move="e2e4"))
        print(f"Reward: {result.reward}, Done: {result.done}")

        # Play until game ends
        while not result.done:
            move = result.observation.legal_moves[0]
            result = await env.step(ChessAction(move=move))

        print(f"Game result: {result.observation.result}")

asyncio.run(main())

For synchronous usage, use the .sync() wrapper:

from chess_env import ChessEnv, ChessAction

with ChessEnv(base_url="http://localhost:8000").sync() as env:
    result = env.reset()
    result = env.step(ChessAction(move="e2e4"))
    print(f"Reward: {result.reward}")

Observation Space

Field	Type	Description
`fen`	str	Board position in FEN notation
`legal_moves`	List[str]	Legal moves in UCI format
`is_check`	bool	Whether current player is in check
`done`	bool	Whether game has ended
`reward`	float	Reward for last action
`result`	str	Game result ("1-0", "0-1", "1/2-1/2")

Action Space

Field	Type	Description
`move`	str	UCI format move (e.g., "e2e4", "e7e8q")

Rewards

Outcome	Reward
Win	+1.0
Loss	-1.0
Draw	0.0
Illegal move	-0.1

Configuration

The environment supports these configuration options:

Parameter	Default	Description
`opponent`	"moonfish"	Opponent type: "moonfish", "random", or None
`opponent_depth`	2	Search depth for moonfish opponent
`max_moves`	500	Maximum half-moves before draw
`agent_color`	None	Agent color: "white", "black", or None (alternate each episode)
`gamma`	0.99	Discount factor for temporal credit assignment

Temporal Discounting

For RL training, the environment computes temporally discounted rewards at episode end. This helps with credit assignment in long games where only the final outcome is known.

When an episode ends, the terminal observation's metadata includes:

discounted_rewards: List of discounted rewards for each agent move
gamma: The discount factor used

The formula is r_t = γ^(T-1-t) × R_final where:

T = total agent moves
t = move index (0-indexed)
R_final = terminal reward (+1, -1, or 0)

Example for a 5-move win with γ=0.99:

Move 0: 0.99^4 × 1.0 = 0.961
Move 1: 0.99^3 × 1.0 = 0.970
Move 2: 0.99^2 × 1.0 = 0.980
Move 3: 0.99^1 × 1.0 = 0.990
Move 4: 0.99^0 × 1.0 = 1.000

Openenv Chess RL Env (Community)