Reinforcement Learning for the Game of Balatro: A Fast Simulator and Two-Policy Training Approach

Reinforcement learning for Balatro is hard because the game combines a 3,000-action state-dependent action space, horizons of several hundred decisions with sparse rewards, and pervasive stochasticity in card draws and shop generation.

Practical examples

• On the main joint run, training metrics show a sharp phase transition: the first-blind clear rate converges above 99% and mean episode return increases by roughly an order of magnitude.
• Single-agent baselines under identical rewards oscillated between greedy and hoarding failure modes and did not stably clear the first blind.

Key takeaways

• Per-agent reward routing with explicit cross-agent credit only partially addresses long-horizon credit assignment.

Research · arXiv cs.AI · Publisher website: arxiv.org · Last verified 12 November 2026 · Catalog ID 133763