Operators running speculative decoding (SD) on sparse mixture-of-experts (MoE) models should not assume the previously predicted concave speedup-vs-batch-size curve. Across 10 sparse MoE models (30B–1T parameters, 8–512 experts), recent fine-grained models show higher speedup at low batch size, so the practical change is to tune and benchmark SD at small batch sizes rather than only at intermediate batches.
