From Dev.to
Speculative Decoding in vLLM: EAGLE, MTP, and Draft Models — When Drafting Pays and When It Costs
Your 70B model generates one token per forward pass. Each pass reads all 140 GB of weights from GPU memory while the arithmetic units sit mostly idle — decode is memory-bound, not compute-bound. Check...
