News
AMD says EAGLE-4 speculative decoding runs on Instinct MI355X GPUs in SGLang and ATOM, with benchmark gains over plain decoding and built-in MTP.
AMD said its EAGLE-4 drafter now runs on Instinct MI355X GPUs from day one in both SGLang and ATOM, positioning the latest speculative-decoding system for lower-latency text generation on the company’s data-center accelerator.
Evidence and context
In a technical article published on 20 October 2026, AMD said EAGLE-4 uses the same checkpoint and launch flags as on other hardware and can run out of the box on the MI355X. The company said it measured EAGLE-Harness cost tables on the GPU so tree sizes fit the device, added tree verification for Qwen3.8’s linear-attention layers, and checked that outputs matched plain decoding up to BF16 rounding at near ties.
AMD’s benchmark table for Qwen/Qwen3.8-27B in BF16 on one MI355X GPU showed one-request throughput of 289 tokens per second in SGLang and 309 tokens per second in ATOM for EAGLE-4, versus 82 tokens per second for plain decoding and 178 tokens per second for the model’s built-in MTP in SGLang. AMD said that amounts to 3.5x plain decoding and 1.6x built-in MTP in its SGLang one-request test, with about 4.7 tokens accepted per check for EAGLE-4 versus about 2.9 for built-in MTP.
The company said the gains held as load increased. On one MI355X GPU, AMD reported EAGLE-4 at 941 tokens per second with four concurrent conversations, 2,167 at 16, and 2,784 at 32 in SGLang; ATOM posted 976, 2,309 and 3,243 tokens per second at those same concurrency levels. AMD said the system shortens its guesses as the GPU fills up so it stays ahead of MTP under load.
AMD also reported scaling across multiple GPUs. On four MI355X GPUs, the company said EAGLE-4 reached 463 tokens per second for one request in SGLang and 489 in ATOM, rising to 4,474 and 4,796 tokens per second respectively at 32 concurrent conversations. AMD said the four-GPU SGLang result was 3.0x plain decoding and 1.5x MTP for one request, and 1.9x and 1.14x at 32 conversations.
The article also described two design changes in EAGLE-4: a single drafter that adapts to batch size and GPU load, and a cheaper low-rank language-model head intended to cut the drafter’s cost by more than 40% without truncating the vocabulary. AMD said the drafter uses one transformer layer so it can fit beside the main model with limited memory.
The company framed the work as an implementation update rather than a broader product announcement. In the same article, AMD said it would like to bring EAGLE-4 to more models and combine it with single-request megakernels for lower latency, but it did not give a date for wider availability or for additional model support.
Credit: Source link
