Efficient Diffusion Is Turning Into a Routing Problem

A video DiT generating a five-second clip is doing attention over tens of thousands of visual tokens, and that attention eats most of the wall clock. A diffusion language model answering a math problem can emit several tokens per round, yet still needs dozens of rounds to finish a paragraph. Both look like compute problems. Both are allocation problems.

The four papers in this cluster each pick a different place to stop spending. One decides which token to commit next. One decides what a distilled student should match. One decides which attention tile deserves full precision. One questions whether you should compress into a latent at all.

The old efficiency playbook is running out

For two years the default move was to make every operation cheaper: fp8 kernels, structured sparsity, smaller models, more aggressive step counts. That works until it doesn't. At some point the accuracy loss shows up as flicker, garbled arithmetic, or mushy textures, and you're stuck tuning a knob that trades quality for latency in a straight line.

These four papers break that line by treating compute as something to route rather than something to shrink. The pattern is consistent: build a cheap predictor of where error will come from, then spend the expensive operations only there.

dSpark: reveal the token that unlocks the rest

Diffusion language models decode in rounds. Each round unmasks a set of positions, and the order you unmask them in shapes everything downstream. The obvious heuristic is confidence: commit the tokens the model is most sure about. dSpark's authors point out why that heuristic is wrong.

A token the model can already predict with high confidence adds almost no information to the context. Revealing a shakier token can make several other positions predictable in the same round. So the schedule that minimizes rounds is not the one that maximizes per-token confidence, it's the one that maximizes the reduction in uncertainty elsewhere.

dSpark builds those schedules offline (an expensive search you only pay for once), then distills them into the model with a KL-divergence regularizer so the student keeps generating coherent text while adopting the faster order.

The numbers are large. On GSM8K, 10.12x speedup over baselines on LLaDA-8B-Instruct and 5.12x on Dream-7B-Instruct, with better task accuracy than the baselines rather than worse. An 8B model already fits on a single 80GB card, so the win here isn't hardware, it's round count. Ten times fewer decoding rounds is the difference between a response that feels instant and one where users assume the tab hung.

One caveat worth flagging: those speedups are measured on math reasoning, a task where early token choices strongly constrain the rest of the derivation. Code generation is also evaluated, but read the per-task breakdown before you assume the 10x travels to open-ended chat.

Quick Take: All four papers replace "make every operation cheaper" with "predict which operations matter, then spend there," and the payoff is measured in multiples rather than percentages.

SF²: when matching the output stops being enough

Distribution Matching Distillation became the default for few-step video because it avoids adversarial training and works across student architectures. It matches the output distribution, which is exactly the problem. There's no explicit constraint on what happens inside the network.

At moderate step counts that's fine. Shrink the step budget aggressively and the student starts producing frames with the right global statistics and the wrong local structure. Fine details smear. Motion develops a low-frequency wobble that no FVD score seems to catch but every viewer sees.

Self Flow Forcing keeps DMD's output objective intact and adds a predictive representation loss on top. The trick is information asymmetry: the student sees a more corrupted input than the exponential moving average teacher, and has to predict the teacher's internal features from that worse starting point. Since the student can't cheat by copying inputs, it has to learn the teacher's representation geometry. No external feature extractor is involved, so there's no DINOv2 dependency and no extra forward pass through a frozen backbone.

The paper reports that this cuts temporal flickering and improves spatial fidelity and motion quality together, and that the mechanism works for both bidirectional and causal video architectures. That matters more than it sounds. Causal models are what you need for streaming video, and most distillation tricks quietly assume bidirectional attention.

Triage: skip, skim, or refine

Attention in video DiTs has an awkward property: the output error from approximation is wildly uneven across tiles. Some tiles can be replaced by their block mean and nobody notices. Others need to be computed but gain nothing from full precision. A small remainder stay error-prone even at low precision, and those are the ones that poison the frame.

Existing sparse attention methods assign one precision to every tile they keep. Triage assigns one of three tiers per tile: skip it (replace with block means), skim it in NVFP4, or refine it with NVFP4 plus residual correction. A lightweight router reads block statistics, predicts the error under each tier, and promotes a tile only when the predicted error reduction justifies the extra compute. One price parameter sets that trade-off, which is the knob you'll actually tune.

A fused kernel executes all three tiers in one pass on the FP4 tensor cores of RTX Blackwell GPUs. Across Wan2.2, HunyuanVideo, and LTX-2.5, Triage speeds up end-to-end generation by up to 2.32x over dense FlashAttention. Here's the part that stands out: in each of five settings it beats every baseline on speed while also landing 0.3 to 3.0 dB higher PSNR than the highest-PSNR baseline. Efficient attention papers usually buy speed with quality. This one claims both.

The catch is hardware. NVFP4 tensor cores mean Blackwell, so this is a bet on RTX 50-series or B200 class GPUs. On H100 or A100 you get none of it.

Key numbers from this cluster

• 2.32x end-to-end video generation speedup from Triage, with 0.3 to 3.0 dB higher PSNR than the best baseline.
• 10.12x decoding speedup for dSpark on LLaDA-8B-Instruct at GSM8K, 5.12x on Dream-7B-Instruct.
• 3 approximation tiers per attention tile, all executed in 1 fused kernel pass.

HiFiPix: pixel space, with the artifacts handled

Real-world super-resolution through latent diffusion hits a ceiling that has nothing to do with the denoiser. The VAE throws away information before diffusion ever starts. Reconstruct fine text on a sign, individual strands of hair, sensor noise patterns, and the latent bottleneck caps how faithful the result can be no matter how good the sampler is.

Pixel-space diffusion avoids that compression entirely. The problem is that pretrained pixel-space models were never built for Real-ISR: they produce visible patch boundary artifacts, and their generative capability is weaker than the latent models you'd be comparing against.

HiFiPix attacks both. For boundary artifacts, it penalizes unrealistic high-frequency periodic components with a Cross-Scale Wavelet Discriminator at the image level, plus a Mixup Contrastive Self-Distillation loss on intermediate representations. The framing is right: patch seams are periodic, so the discriminator should look at periodic structure rather than at whole-image realism.

The more interesting finding is that the generation-fidelity trade-off is not uniform across LoRA layers. Some adapters push generative detail, others drag reconstruction fidelity down. Selective LoRA Pruning finds that split and prunes adaptively, so you keep generative capability without giving back accuracy on the reconstruction side.

The authors claim consistent wins over latent-space state of the art on restoration accuracy and perceptual quality, and promise code and models. Treat that as a promise until numbers show up. Pixel-space diffusion at high resolution is expensive in memory and steps, and the abstract doesn't say what it costs.

Where the four land on the same map

Paper	What it stops paying for	Training needed	Reported gain	Hardware catch
dSpark	Decoding rounds in diffusion LMs	Yes, schedule distillation with KL regularizer	10.12x (LLaDA-8B), 5.12x (Dream-7B) on GSM8K	None specific, 8B fits one 80GB GPU
SF²	Steps in video distillation	Yes, DMD plus representation loss	Less flicker, better spatial and motion quality	None specific, no external feature extractor
Triage	Full-precision attention tiles	No, training-free	Up to 2.32x end-to-end, +0.3 to 3.0 dB PSNR	Requires Blackwell NVFP4 tensor cores
HiFiPix	VAE compression before diffusion	Yes, LoRA training plus pruning	Beats latent-space Real-ISR per authors	Pixel-space cost not reported

Those three bars do not share a denominator, and that's the first thing to fix in your head before quoting any of them. The dSpark numbers count decoding rounds on a math benchmark. The Triage number is wall-clock end-to-end video generation. A 2.32x on video is worth more GPU-hours than a 10x on text rounds, because video inference is where the money actually goes.

Common pitfalls

Don't quote speedup factors across papers without checking what's in the denominator. Decoding rounds, attention FLOPs, and end-to-end latency produce numbers that differ by an order of magnitude for the same underlying improvement. If a vendor slide says "10x faster," ask what the timer was started on.

Don't swap a confidence-based unmasking schedule into a diffusion LM and assume you're getting dSpark's numbers. The gain comes from the learned schedule, not from the sampler change. Reordering by confidence is what the baselines already do.

Don't benchmark Triage on pre-Blackwell hardware. The three-tier scheme depends on NVFP4 tensor cores and a single fused pass; on an H100 you either fall back to a slower path or get nothing. Also profile end-to-end rather than attention-only, since the 2.32x figure depends on attention being the dominant cost in your particular pipeline.

Don't judge a distilled video model on frame-level PSNR alone. Output-space matching can look healthy on aggregate metrics while the clip flickers in a way viewers find unacceptable. SF² exists because that gap is real. Sample consecutive frames and look at them.

Don't treat pixel-space super-resolution as a drop-in replacement for your latent pipeline. The memory profile, the step count, and the failure modes are all different, and patch boundary artifacts show up as periodic texture that a whole-image metric will happily ignore.

The constraint that binds all four

Every one of these methods moves the decision into something cheap and makes it once. dSpark pays for schedule search offline. Triage pays for router calibration and then reads block statistics at runtime. SF² pays for a teacher representation target at training time. HiFiPix pays for a pruning pass over LoRA layers.

The expensive thing is decided in advance and amortized across every request. That, not any single speedup factor, is what transfers if you're building your own efficient diffusion stack. Look for the decision you're currently making per-token, per-tile, or per-step, and figure out whether a cheap predictor could make it for you.

Where to spend your GPU budget

If you're serving a diffusion language model and your latency is dominated by round count, schedule distillation is the change with the largest multiple attached to it. On LLaDA-8B the reported 10x on GSM8K comes with better accuracy, not worse, which is rare enough that it's worth reproducing before you dismiss it.

If your video DiT runs on Blackwell and attention dominates your profile, Triage is training-free and the only one of these four you can try this week. If you're on H100 or A100, skip it entirely and put the effort into step distillation instead, where the gains don't depend on FP4 tensor cores.

If you're distilling a video model below roughly eight steps, assume output-only distribution matching will flicker and budget for a representation term. And keep an eye on HiFiPix: pixel-space Real-ISR is where the reconstruction ceiling actually moves, but wait for the code and the latency numbers before you plan a migration.

Papers referenced

• Learning When to Commit: Schedule Distillation for Fast Diffusion Language Models: http://arxiv.org/abs/2610.35804v1
• Beyond Distribution Matching: Self-Supervised Representation Forcing for Few-Step Video Generation: http://arxiv.org/abs/2610.35594v1
• Triage: Skip, Skim, or Refine Attention Tiles in Video Diffusion Transformers: http://arxiv.org/abs/2610.35706v1
• HiFiPix: Bringing High-Fidelity Real-World Super-Resolution Back to Pixel Space: http://arxiv.org/abs/2610.35515v1