Tencent Open-Sources AngelSpec: A Unified Training Framework for MTP and Block-Parallel Speculative Decoding on Hy3 Models

Tencent has released AngelSpec, an open-source, torch-native training framework for speculative-decoding draft models. The release covers both autoregressive multi-token prediction (MTP) and the block-parallel DFlash family. Most speculative-decoding work searches for one drafter that scores well on an averaged benchmark mixture. Real serving traffic does not look like that mixture. AngelSpec treats workload heterogeneity as a first-class design constraint, and specializes structure, training data, and verification depth around it. Why one universal drafter underperforms Speculative decoding is lossless. A lightweight drafter proposes several future tokens, and the target model verifies them together in one forward pass using rejection sampling. Acceleration then depends on two things: how many draft tokens get accepted, and how long the complete draft–verify round takes. Those two quantities move in opposite directions across domains. In high-entropy open-ended conversation, many continuations are semantically valid. The target may pick any one of them, so acceptance decays quickly with proposal depth. Generating and verifying a long block wastes compute. Autoregressive MTP drafting fits this regime because it proposes a shorter candidate sequence. Code and mathematical reasoning behave differently. Programming syntax, repeated identifiers, formal expressions, and step-by-step derivations constrain future tokens more strongly. These workloads create longer predictable spans, which is exactly what block-parallel drafting amortizes well. AngelSpec therefore ships two complementary drafters, not one compromise. The MTP model is trained on rich, diverse conversation-oriented data. The block-diffusion model is strengthened with code- and mathematics-focused samples. The MTP path: Training-Time Test and target-model rollout The original Hy3 model is trained with a single MTP layer and no recurrent self-conditioned unrolling. At inference the block can be reused recurrently, but the training objective never prepared it for a long self-generated chain. Errors accumulate with depth, so the second and third draft positions accept at substantially lower rates than the first. AngelSpec addresses this train–inference mismatch with a shared-parameter, multi-depth scheme. It retains D logical prediction depths but reuses one physical MTP block. During training that block is autoregressively unrolled for D steps, with each prediction fed into the next invocation. Following the Training-Time Test principle of EAGLE-3, depth k+1 receives the arg max prediction from depth k instead of the ground-truth token. Parameters are shared, but supervision stays depth-specific: the teacher target advances one future position at every depth. Two further choices carry most of the gain. First, the target backbone and target language-model head are frozen, and MTP inputs from the backbone are detached. The drafter improves its proposal distribution without touching the distribution that verification must preserve. Second, training uses target-model rollout: responses are generated by the frozen target rather than taken from the original reference corpus. That produces the exact token choices, hidden-state trajectories, and local uncertainty patterns MTP has to approximate at serving time. The measured effect is concentrated where it should be. At T = 0, mean acceptance moves from 52.8% to 66.4%, and mean accepted length from 2.58 to 2.99. The first-position rate is nearly preserved at 0.799 → 0.814. The deeper positions carry the delta: p3 climbs from 0.290 to 0.706 on GSM8K, and from 0.387 to 0.757 on HumanEval. (function(){ window.addEventListener("message", function(e){ if (!e.data || e.data.type !== "mtp-resize-asx1") return; var f = document.getElementById("mtp-angelspec-bench"); if (f && e.data.height) f.style.height = e.data.height + "px"; }); })(); DFly: hybrid target conditioning and a