Appearance
Three ways to break the autoregressive bottleneck
Autoregressive decoding has one hard constraint: token n+1 cannot start until token n finishes. Every trick in a modern serving stack, from KV caching to speculative decoding, works around that. Three papers from this cycle go after the constraint itself, and each one saws through the chain in a different place.
They are not variations on one idea. One replaces the token-by-token inner loop with a continuous flow. One reuses weights to buy depth without buying parameters. One keeps a set of candidate tokens alive per position instead of committing early. Read together, they map the space of what "not autoregressive" can actually mean.
Three places to cut the chain
A single forward pass is already parallel. The serialization lives across positions, and across the sequence of decoding steps. So the question is where else to find parallelism.
You can parallelize inside a block of tokens. Generate the block bidirectionally, condition it causally on everything before it, and accept that within the block the tokens are resolved together rather than one at a time.
You can parallelize across depth. Apply the same layers repeatedly, so a small parameter count buys a large effective depth, and treat the extra passes as iterative refinement.
You can parallelize across positions with an any-order process. Keep uncertainty explicit, refine it over many steps, and let the model change its mind about token 12 after seeing token 40.
The three papers below are one example of each.
Lafa: flow matching inside a block
Lafa is a blockwise flow language model scaled to 8B parameters. Causal attention runs on the prefix, bidirectional asynchronous flow runs inside each block. That split is the whole design: because the prefix is still causal, prefix caching works, so you don't pay to re-encode context on every step. Because the interior of a block is bidirectional, you can decode the block in parallel.
An 8B model in bf16 is 16GB of weights, so memory isn't the binding constraint. Decode speed is. That's the problem Lafa targets.
The training budget is the surprising part. Lafa was trained at 0.5B, 7B, and 8B with fewer than 30B tokens. An 8B model trained from scratch normally sees trillions. Under 30B tokens only makes sense because the recipe starts from a pretrained autoregressive checkpoint and adapts it, so the flow objective is teaching a new decoding behavior to a model that already knows language.
Flow training on a pretrained AR model preserves or improves its capabilities, which the authors attribute to a multi-token prediction effect. That observation turns into the deployment story. Because the flow path and the AR path live in the same weights, you can let the flow head draft several tokens and let the AR path verify them autoregressively. One model, two roles, self-speculative decoding.
The result is 2.54x the throughput of an AR model at comparable quality, ahead of multi-token prediction and discrete diffusion baselines. In serving terms: the same accelerator serves two and a half times as many users against the same quality bar, assuming draft acceptance holds up on your traffic. Lafa can also be distilled into flow maps for one- and few-step generation, which trades some quality for another jump in latency.
Quick Take: all three papers target the same serialization cost, but only Lafa currently looks like a drop-in serving win, and it gets there by keeping the pretrained AR weights instead of discarding them.
AsymLoop: stop re-deriving your context
Looped transformers apply a shared stack of layers repeatedly. You get more effective depth without more parameters. The problem is that full-stack looping puts two different jobs into the same recurrent computation: building the context from the prompt, and refining an intermediate state toward an answer. Those jobs have different optimal amounts of iteration. Context building is a one-shot operation. Refinement isn't.
The Asymmetric Looped Transformer separates them. A single-pass causal encoder produces layer-specific global key-value caches that stay fixed across iterations. A weight-tied recurrent decoder produces local KV from its own evolving state. Joint attention over both sources under a single softmax lets each update mix persistent context with the current intermediate state.
The practical consequence is that the expensive part runs once. Prompt encoding happens on the first pass; the loop only refines the decoder state, and the global KV is fixed, so looping does not multiply context memory the way a naive re-encode would. Iteration costs compute per token, not memory per token.
Training this needs care. The paper applies a heavy-tail-guided layerwise learning-rate method to the encoder and decoder. Without a schedule that treats the two halves differently, one of them dominates the gradient and the separation collapses back into ordinary looping.
Results on FineWeb-Edu across four scales from 60M to 1B show lower validation perplexity and higher downstream accuracy than a full loop at matched effective depth and matched token budget. At 1B, average zero-shot accuracy over seven commonsense reasoning benchmarks moves from 49.58% to 50.85%. That is 1.27 points, roughly 2.6% relative, for zero additional parameters. The gain is small in absolute terms, and that's the point: looped models usually buy compute savings by giving up accuracy, and here you keep the savings and gain a little.
Binary diffusion: keeping several answers alive
Masked diffusion replaces a token with [MASK]. Uniform diffusion resamples it from the vocabulary. Both destroy a token's identity in a single corruption event, which means the noisy state cannot say "this position is either a 3 or a 7 and nothing else." That's a problem when the task is reasoning, because mid-solve, "either 3 or 7" is exactly the state you're in.
Binary Discrete Diffusion makes the state a set. Each position carries a binary membership vector over the vocabulary, and the corruption process adds and removes candidates rather than obliterating the token. Uncertainty degrades gradually and can be recovered. The reverse model learns to add and remove, so it can widen a set when evidence is thin and narrow it when constraints bite.
On Sudoku-Extreme, Boxoban, and graph coloring, the symmetric formulation beats well-tuned masked and uniform baselines under matched architectures and matched training, sampling, and tuning budgets. The largest gains land on the hardest instances, which is where you'd want them.
| Task domain | Why candidate sets fit | Reported outcome |
|---|---|---|
| Sudoku-Extreme | Each cell holds a set of legal digits and shrinks under constraint propagation | Beats masked and uniform diffusion, biggest margin on the hardest puzzles |
| Boxoban | Box positions are constrained jointly, so early commits are expensive | Beats masked and uniform diffusion baselines |
| Graph coloring | Color choice at one node depends on neighbors resolved later | Beats masked and uniform diffusion baselines |
| Text8, TinyStories, OpenWebText | Language modeling has high entropy per position but weak hard constraints | Competitive, not better than baselines |
The last row matters. This is a reasoning-state improvement, not a language-modeling throughput win. If you read the abstract looking for a faster text generator, you're reading the wrong paper.
Where the three designs actually differ
The papers optimize for different things, and the comparison only makes sense if you line up the tradeoff each one is making.
| Lafa (blockwise flow) | AsymLoop | Binary diffusion | |
|---|---|---|---|
| Generation order | Causal across blocks, bidirectional inside a block | Autoregressive, extra compute per token through looping | Any-order over many refinement steps |
| Reuses pretrained AR weights | Yes, this is the core of the method | No, trained from scratch in the study | No |
| Scales tested | 0.5B, 7B, 8B | 60M, 130M, 350M, 1B (four scales) | Task-specific and language-modeling runs |
| Training cost | Under 30B tokens | Matched token budgets, more compute per token | Controlled budgets matched to baselines |
| Headline number | 2.54x decode throughput at comparable quality | 49.58% to 50.85% average zero-shot at 1B | Win on Sudoku-Extreme, Boxoban, graph coloring |
| What it costs you | Draft acceptance risk, new serving machinery | Higher compute per token, encoder/decoder LR tuning | Many sampling steps, worse text perplexity |
One pattern stands out. Lafa is the only one of the three that treats an existing checkpoint as an asset rather than a starting point to be overwritten. That is why its training budget is two orders of magnitude smaller than what the other two approaches imply. If your organization already owns a pretrained model, that asymmetry changes the calculus more than any benchmark delta.
Key numbers from the three papers
8B parameters: Lafa's largest scale, trained on fewer than 30B tokens. 2.54x: decode throughput versus an AR baseline at comparable quality. 1.27 points: AsymLoop's average zero-shot gain at 1B (49.58% to 50.85%). 0: extra parameters AsymLoop spends to get that gain. 1: corruption events masked diffusion needs to erase a token's identity.
What trips people up
Converting a checkpoint is not the same as pretraining one. Lafa's under-30B-token budget works because it starts from a pretrained AR model that already knows language. If you initialize from scratch and train 30B tokens on a flow objective, you get something worse than a small AR baseline. The recipe is a conversion, and a conversion needs something to convert.
The 2.54x is not a constant you can put in a capacity plan. It comes from self-speculative decoding, where flow drafts are verified by the AR path. On high-entropy outputs like open-ended code or unusual formatting, draft acceptance drops and the multiplier slides toward 1. Measure acceptance rate on your own traffic before you re-plan cluster capacity around it.
Looping the whole stack and expecting AsymLoop's numbers is a common misread. The gain comes from separating the one-shot encoder from the recurrent decoder. Apply the same recurrent block to context and state, and you've rebuilt the Full Loop baseline, which the paper shows is worse at matched effective depth. The asymmetry is the method.
Screening binary diffusion with validation loss on text will cause you to throw it away. Its language-modeling numbers are competitive at best, while its constraint-satisfaction numbers are better than tuned baselines. If your selection metric is bits-per-byte on Text8, you'll eliminate the architecture that solves your scheduling problem before it gets a fair hearing.
One thing to remember
One thing to remember: every number in these papers measures a narrow slice of the pipeline. Lafa measures decode throughput at matched quality on its own evaluation set. AsymLoop measures average zero-shot accuracy at 1B across seven benchmarks. Binary diffusion measures solve rates on constraint puzzles. None of them measures what happens when the model sits behind a product with a 500ms budget, adversarial prompts, and a cache hit rate of 30%. Read the numbers as existence proofs for a design direction, not as service-level guarantees.
What this means in practice
If you already own a pretrained AR checkpoint and your bottleneck is decode throughput, Lafa's conversion path is the cheapest experiment on this list. Under 30B tokens of adaptation plus a self-speculative decoder gets you a 2.54x ceiling, and the AR path stays in the loop as the verifier, so a bad draft costs latency rather than quality.
If you're training a small model from scratch and parameters are the binding constraint, AsymLoop is the better trade. You pay extra compute per token for the loop and collect about 1.3 points of zero-shot accuracy at 1B with an identical parameter count, plus lower validation perplexity at every scale tested.
If your problem is constraint satisfaction, whether that's scheduling, puzzle solving, or structured planning, set the throughput arguments aside and test candidate-set diffusion. Binary diffusion wins where partial uncertainty is genuinely the state of the problem, and it stays competitive on plain text, so you aren't surrendering general capability to get the reasoning behavior.