Denoising is where the money goes

A 1024x1024 image through SDXL runs 25 to 50 denoising steps, and every step touches the full latent grid. That's the bill. For two years the standard ways to shrink it were distillation and step reduction. Both work, and both cost you something in quality or fidelity.

A batch of recent papers attacks the problem inside the diffusion process itself. The common thread: the model already produces signals about which regions of the latent are settled and which are still moving. Read those signals and spend compute accordingly.

Token merging, batch diversity, RL alignment, and continuous language diffusion each get a version of that treatment here. Six of the seven methods are training-free, and the two that aren't change the training loop rather than the inference path.

Token merging, grounded in diffusion dynamics

Token merging groups similar tokens and runs attention on a representative, cutting compute roughly in proportion to the compression ratio. It drops into any existing pipeline. The failure mode is easy to picture: merge a background tree trunk with the shirt of the person named in your prompt, and the trunk dissolves into noise.

Earlier methods protected prompt-aligned foreground and merged aggressively elsewhere. A text prompt is a bad map of what matters in an image. Contact shadows and fine textures that never appear in your string still decide whether the output reads as coherent.

DaTMe's contribution is deciding what is safe to merge using the model's own trajectory. The central quantity is the Tweedie denoised estimate, the model's best guess at the clean image given the noisy latent at step t. Two signals come out of it. The temporal signal measures how much that estimate changes from step t to t+1, so tokens still shifting are unresolved and get preserved. The structural signal measures local high-frequency content (boundaries, fine detail) and protects tokens that carry it.

Tokens that are both temporally stable and structurally homogeneous are safe merge candidates. You set a compression ratio, and the framework assigns the roles.

On PixArt-α and FLUX, the importance map holds fidelity across the whole image rather than just the prompted subject. That distinction is the point, because prompt-aligned merging tends to fail precisely in the region outside the prompt.

Language diffusion gets a purpose-built backbone

Continuous diffusion language models generate text by denoising in a continuous space instead of predicting the next token. Parallel generation is the appeal. The problem is that most implementations borrow a transformer from either a causal LLM or an image Diffusion Transformer, and neither was designed for this job.

LangDiT is a systematic study of what actually helps, and it lands on four changes: feature and query-key normalization, wider noise conditioning, likelihood-shortcut gradient routing with calibration, and confidence-gated training self-conditioning.

On unconditional 1024-token OpenWebText generation, LangDiT hits 21.72 GenPPL at 1024 NFEs and 44.72 at 128 NFEs with the LangFlow formulation. Adapted to Plaid-style, it reaches 11.33 GenPPL at 1024 NFEs and 27.51 at 128 NFEs.

A GenPPL of 11.33 at 1024 NFEs puts continuous language diffusion in the neighborhood of small autoregressive models, which is new. The 128-NFE number is the one that matters for production. You trade roughly 2.4x worse perplexity for 8x less compute, and 128 forward passes per 1024-token sequence (about 750 words) is the range where this stops being a research curiosity.

The gains hold while keeping each model's underlying representation, training objective, and inference sampler fixed. Architecture alone accounts for the improvement.

Quick Take: across all of this work, diffusion efficiency gains now come from deciding where compute is unnecessary, and the models themselves are telling you where that is.

Diversity is a collective property of the batch

Strong classifier-free guidance pushes every sample toward the high-density mode of the conditional distribution. If you've ever set guidance scale to 12 and generated 64 images, you've probably gotten back 64 variations of the same photo. For data augmentation that's a disaster: 64x the compute for one useful sample.

Existing fixes use sum-of-pairwise-distance or repulsion objectives, and they respond weakly to the actual failure. A set of points can keep every pairwise distance large and still collapse onto a one-dimensional subspace. Diversity lives at the set level, not the pair level.

DetNO formulates diverse generation as RKHS volume maximization on the initial noise. It maximizes the log-determinant of an RBF Gram matrix over one-step Tweedie predictions. That quantity equals the squared batch volume, and its repulsion is strongest exactly when the batch is most collapsed, then self-attenuates as diversity improves.

An anchor-based fidelity term keeps the optimized noises near plausible regions, and Lagrangian relaxation collapses the objective to two terms with a single hyperparameter, λ, governing the diversity-quality trade-off. You set λ and change nothing else. No training, no sampler modification.

Across SD1.5, SDXL, and SD3, DetNO leads on VendiScore and Mean Self-Similarity. A downstream few-shot classification experiment found statistically significant accuracy gains in the low-shot regime, which is the whole reason you'd generate synthetic data in the first place. With the pipeline fixed, both RBF-based aggregators beat InfoNCE, and the log-determinant scores best on both benchmarks. The advantage over InfoNCE widens as the batch grows.

RL alignment without the rollout tax

Reward-based fine-tuning for diffusion usually works by perturbing the reverse denoising trajectory with noise. That gives you a stochastic policy and a costly one, since you sample the whole trajectory before you can score anything.

Routing Policy Optimization asks whether the exploration can happen inside the generator while the original sampler stays intact. The answer is attention routing. At a handful of selected denoising levels, each query block keeps its own block and samples additional key blocks in proportion to the generator's native attention mass, or falls back to dense attention. The sampled route perturbs the transition velocity, and a Gaussian likelihood around the dense velocity scores that perturbation with a score-function objective.

The estimator is a controlled two-rollout setup. Both rollouts share the same prompt, initial latent, routed denoising levels, and dense anchor layers. They differ only in their sampled attention routes, and the reward difference between them supplies the advantage signal. Because the likelihood differentiates the dense velocity itself, the learned updates stay useful once routing is removed.

At inference you restore dense attention and keep the deterministic sampler. Nothing on your serving path changes.

The reported result is nearly 4x higher training efficiency than Flow-GRPO-style methods on text-to-image and text-to-video. Four times less compute is the difference between a reward function you tune overnight and one you tune before lunch.

Prompt rewriting at span granularity

Prompt rewriters improve frozen text-to-image models by rewriting the prompt before generation. Existing RL methods apply one image-derived reward to every subword token. But "two cats on a blue rug" is not a bag of tokens. The object, the count, the attribute, and the relation are coherent spans, and credit should flow along those spans.

SCOPE builds on Group Relative Policy Optimization. The rewriter generates an enhanced prompt and its semantic partition in the same pass, then pools shifts in policy log probabilities within each span to derive a shared coefficient for that span's token losses, keeping a single advantage per completion. No separate span rewards are needed, and no extra image renders.

Across Qwen-Image, SD3, and FLUX.1-dev, SCOPE gets 9.48% relative improvement over vanilla GRPO on GeneVal and 7.62% on T2I-CompBench. On object-counting benchmarks, that margin is the difference between a rewriter that drops attributions and one that gets the count right most of the time.

The ablations show both semantic boundaries and within-span pooling contribute, which suggests semantic structure is a useful inductive bias for RL-driven prompt engineering beyond this one setting.

Method	Hooks into	Training-free	Core mechanism	Reported result
DaTMe	Image diffusion tokens	Yes	Tweedie-based importance map	Fidelity held on PixArt-α, FLUX
LangDiT	CDLM backbone	Yes (drop-in)	Normalization, noise conditioning, gradient routing	11.33 GenPPL at 1024 NFE
DetNO	Batch noise init	Yes	Log-determinant volume maximization	Best VendiScore on SD1.5, SDXL, SD3
RPO	RL training loop	No	Stochastic attention routing	~4x training efficiency vs Flow-GRPO
SCOPE	Prompt rewriter	No	Span-level credit pooling	+9.48% GeneVal, +7.62% T2I-CompBench
EOS-Story	Storytelling pipeline	Yes	Cross-contextual bridging	Competitive on both Consistory+ variants
SIRS	Masked diffusion sampler	Yes	Single-increment rejection	One query per step, bounded KL

Storytelling that adapts to the scene

Generating a coherent image sequence from a narrative prompt is harder than generating one image. Existing methods lean on identity preservation and break when the scene changes: the character looks right while the framing, lighting, and setting ignore the story.

EOS-Story is training-free and splits the problem across text and vision. In the text space, it decouples the narrative into identity and scene sub-prompts so neither dilutes the other. That leaves the scene without appearance cues, so Cross-Contextual Bridging injects a phrase-level appearance residual into the scene's <EOS> embedding.

In the vision space, matching identity and scene subjects at different scales causes a small set of identity tokens to be transferred repeatedly, which produces structural artifacts. Adaptive Identity Preservation anchors identity features at the scene-specific scale and integrates them through Identity-Aware Self-Attention.

The authors extend the Consistory+ benchmark with varied framing scenarios (Consistory+_+). EOS-Story matches or beats prior work on both, without any fine-tuning.

Parallel masked decoding, bias-corrected

Masked diffusion models speed up generation by unmasking several tokens at once. Sampling those tokens independently ignores their correlations and biases the output. Anari et al. corrected the bias with rejection sampling, decomposing the log acceptance probability into a sum of token-level terms. The cost was one score query per unmasked token, which eats into the parallel speedup you were chasing.

SIRS estimates the same log acceptance probability from a single randomly sampled increment, at the cost of one query. It converts a Poisson number of clipped estimates into accept/reject decisions through a Bernoulli factory. Each query returns conditionals for all remaining tokens in the block, so SIRS averages each increment over those tokens at no extra cost, which cuts the clipping rate.

The guarantee: the KL divergence between the data distribution and the SIRS output is bounded by a clipping term plus the model's training error. With exact scores, parallel decoding pays the full within-block dependence, and SIRS pays only the part beyond the clipping range. One query per step instead of one per token is the difference between parallel unmasking saving you time and parallel unmasking saving you nothing.

Key numbers

GenPPL of 11.33 at 1024 NFEs is where continuous language diffusion starts looking like a real alternative to autoregressive generation.

27.51 GenPPL at 128 NFEs is the operating point to watch: 8x less compute for roughly 2.4x worse perplexity.

9.48% relative gain on GeneVal comes from assigning credit along semantic spans instead of per token.

7.62% on T2I-CompBench under the same setup, across three different frozen generators.

Nearly 4x training efficiency over Flow-GRPO-style alignment, because exploration happens in attention routing rather than in trajectory perturbation.

A single λ controls the DetNO diversity-quality trade-off. No retraining, no sampler changes.

Common pitfalls

Don't apply token merging at a fixed compression ratio across every denoising step. Early steps are where structure is still forming, and aggressive merging at t=0 smooths away the boundaries the structural signal is trying to protect. Ramp the ratio instead.

Don't chase batch diversity with pairwise repulsion. Two images can sit far apart in pixel space and represent the same mode. A set-level objective, like the log-determinant DetNO uses, is the only one that responds to collapse onto a subspace.

Don't judge a continuous diffusion language model by its 1024-NFE perplexity. That number looks great and takes forever. Test at 128 NFEs, because that's the budget you'll actually ship, and 44.72 GenPPL is a very different product than 21.72.

Don't leave RPO's attention routing on at inference. Routing is a training-time exploration action. If you keep it active during serving, you've replaced a deterministic sampler with a stochastic one and the reward-quality gains evaporate.

Don't pool prompt-rewriter credit across the whole completion. A single advantage for every token can't tell the rewriter which span of the prompt fixed the object count, which is exactly the failure SCOPE was built to avoid.

One thing to remember

Nearly every efficiency gain in this batch comes from information the diffusion model already produces while denoising. The Tweedie estimate, the attention mass, the inter-step delta: none of it needs a new head, a new loss, or a new sampler. If you're shopping for speedups, read your own intermediate tensors before you reach for a bigger GPU.

The bottom line

If you're serving high-resolution text-to-image and paying for every denoising step, bolt DaTMe onto your existing PixArt-α or FLUX pipeline. It's training-free, and the importance map rides on the Tweedie estimate you already compute.

If you're generating synthetic training data and your batches keep collapsing to the same prototype, add DetNO's log-determinant objective to your noise initialization. Fix the model and the sampler, change only the batch objective, and the diversity advantage widens as your batch grows. Watch the 128-NFE operating point for continuous language diffusion over the next year, since LangDiT just moved that number out of the toy range.

If you're aligning a diffusion model with RL and the rollout loop is your bottleneck, look at RPO. Nearly 4x training efficiency over Flow-GRPO-style methods turns reward-function iteration into an afternoon task instead of an overnight one.