TableTune is a system for dynamic speculative decoding in vLLM that replaces a hand-written draft-length table with one that is measured and self-correcting. The best draft length—the number of tokens a small drafter guesses per step—shrinks as the batch grows, because speculative decoding only helps while the GPU has spare compute.
