The bill comes from the layers, not the image

A 1024x1024 image through a patch-14 vision encoder produces about 5,300 patches. Most multimodal LLMs project that down by 4x or 9x before it reaches the language model, which leaves somewhere between 600 and 1,300 visual tokens in the context.

That count isn't the problem. The problem is that it gets re-read at every decoder layer, for every generated token. A 30-layer model holding 1,000 visual tokens does roughly 30,000 token-layer passes over image content before it writes a single word, and most of those tokens are sky, asphalt, or a blank wall.

Two fixes exist. Drop tokens, or make them cheaper. All four papers here work the first angle, and each one runs into the same wall: the set of visual tokens that matter is not fixed. It shifts as the query moves through the decoder, and it shifts again as the reasoning chain gets longer.

CatchUp: pruning that admits it made a mistake

Query-conditioned pruning scores visual tokens by their attention from the text, keeps the top-k, deletes the rest. On paper it holds up. In practice I've watched it slice a road sign out of a street scene at layer 3 and never get it back, which is precisely the moment the user asks about the sign.

CatchUp starts from a sharper observation. Recoverable routing already exists, where skipped tokens get parked for later reconsideration, but those tokens come back with stale representations. They bypassed five or ten decoder layers of transformation and get re-admitted as if nothing happened. Feed the current-depth oracle states instead of the stale ones and grounding jumps.

So CatchUp re-evaluates query-conditioned relevance at each decoder interval, processes only the selected visual tokens, parks the rest, and on re-admission approximates the missed representation evolution using transition signals from semantically and spatially related tokens. No training. No new weights.

The numbers: at a 64-token budget, CatchUp keeps 92.17% of full-model performance averaged across six multimodal benchmarks while activating 88.9% fewer visual tokens, and end-to-end latency drops 8.0%. It takes the highest average relative retention at 64, 128, and 192 tokens across both benchmark groups, including three referring-expression grounding datasets.

Read those two numbers together and they tell you something uncomfortable. An 88.9% cut in active visual tokens buys 8.0% in wall clock. Attention over visual tokens is a fraction of your decode cost. MLP blocks, KV cache traffic, and sampling eat the rest. If you're budgeting a pruning rollout on the token count, you'll be disappointed in p99.

Quick Take: Each of these papers is really about recovering information an efficiency trick threw away, and the ones that quantify the recovery rather than the savings are the ones you should believe.

The Repeat Curse in diffusion decoders

Diffusion-based MLLMs decode by iterative denoising with bidirectional attention. That design blocks the standard autoregressive KV cache, so production deployments fall back on approximate caching: reuse intermediate states across denoising steps. The approximation has a failure mode where a token repeats at adjacent positions, producing a run that breaks the sentence.

The CoTA++ authors traced it through the information flow and found three separate disruptions. Within a step, caching disperses the attention that anchors a position on nearby context tokens. Across steps, similarity-based refresh has no notion of staleness, so some positions decode against key-value states that no longer describe anything. Across layers, the deep-layer entropy of the context fails to converge exactly where the repetition shows up.

Their fix is training-free and composes with whatever cache you already run:

• Context-Token Attention Re-anchoring rebuilds the attention routing against the current context.
• Decode-Aware Refresh reserves part of the refresh budget for positions about to be committed.
• Context-Token Entropy-Guided Voting discounts a position's confidence by the deep-layer entropy of its context, which reorders the unmasking.

The three act at non-overlapping stages of a single decoding step. Across three dMLLMs and two caching backends, CoTA++ cuts the sample repetition rate on long responses from as high as 99.6% down to at most 17.0%, recovers most of the accuracy that caching destroyed on nine benchmarks, and holds the caching speedup while adding under 15% overhead.

17.0% is still bad in absolute terms. It's the difference between a response that's unusable and one that needs a retry heuristic. If someone ships a diffusion MLLM to you in the next year, ask what their repetition rate is before you ask about throughput.

Xiaomi-Lens-0: the mask comes out of the language model

Whole-image question answering is a solved enough problem that a 4B model handles it. Region-level understanding is not, and one reason is supervision: text-only training never specifies which pixels support an answer.

Xiaomi-Lens-0 is a 4B model built on Qwen3.5 that predicts masks from its own visual representations. A lightweight decoder combines vision-encoder patch features with the language model's mask-token states, so mask supervision reaches the shared backbone without a separate segmentation encoder. Each latent decodes to a mask, which means the model can interleave reasoning with region-specific latents and reason over regions it has already localized. Grounded reasoning traces supervise which regions get visited during chain-of-thought.

Across all eight RefCOCO, RefCOCO+, and RefCOCOg splits, it gains 5.6 points on average over Qwen3.5 in referring comprehension and posts the highest segmentation cIoU among the compared models. It also beats Qwen3.5 on every perception benchmark tested.

4B parameters is roughly 8GB of bf16 weights, which fits on a 24GB card alongside a KV cache and a small image batch. Region-precise grounding stops being a datacenter-only capability at that size.

The part I find more interesting than the mask quality is the removal of the segmentation head. Two-tower designs, where you bolt a SAM-class decoder onto a vision-language model, are a maintenance tax and a latency tax. Collapsing that into the language model's own latents is the right direction.

GeoReason: keeping [SEG] from drifting

Remote sensing makes every efficiency compromise visible. Targets are cluttered, scale variation is extreme, and a parking lot vehicle might be six pixels wide. GeoReason names the failure precisely: under autoregressive generation, the segmentation token [SEG] is emitted at the end of a long reasoning chain, and by then the visual evidence behind it has been diluted by linguistic abstraction and accumulated decoding noise. The authors call it sequential alienation. Marginal fidelity loss on a six-pixel target means total localization failure.

Their fix moves the token. An Anticipatory Prior injects [SEG] directly into the initial user query, converting it from a terminal output into a primary condition, so it acts as a persistent spatial anchor instead of a late afterthought.

A Location-aware CoT enforces coarse-to-fine localization using bounding boxes, and a Latent Patch-to-Token Alignment loss explicitly ties the latent segmentation token to physical remote sensing textures. Experiments report consistent wins on Earthreason and RRSIS-D.

That's the whole claim. No numbers in the abstract, no cIoU, no per-dataset deltas. For a paper whose entire thesis is about spatial fidelity, the absence of quantified fidelity numbers is the thing I'd want fixed before citing it.

Key numbers88.9% fewer active visual tokens buys 8.0% lower end-to-end latency (CatchUp, 64-token budget, six benchmarks). 99.6% → 17.0% worst-case repetition rate on long responses (CoTA++, three dMLLMs, two cache backends). +5.6 points average referring comprehension across eight RefCOCO splits from a 4B model (Xiaomi-Lens-0 over Qwen3.5). Zero benchmark figures in the GeoReason abstract, on the one paper where spatial precision is the entire point.

Side by side

Paper	What it attacks	Training	Overhead reported	Headline number
CatchUp	Visual token count across decoder depth	None, training-free	8.0% lower latency at a 64-token budget	92.17% average retention with 88.9% fewer active visual tokens
CoTA++	Approximate KV caching in diffusion MLLMs	None, training-free	Under 15% of the caching speedup	Repetition rate 99.6% to 17.0%
Xiaomi-Lens-0	Region grounding supervision	Full training of a 4B model	Not reported	+5.6 points average across 8 RefCOCO splits
GeoReason	[SEG] drift in long reasoning chains	Full training	Not reported	Wins on Earthreason and RRSIS-D; no published figures

The split is clean. The two training-free methods report their overhead honestly, including the case where the overhead is most of the gain. The two trained models report benchmark deltas and stay quiet about cost.

What it's like to wire these in

I pulled query-conditioned token pruning into a document QA service last quarter because our prefill time on scanned pages was embarrassing. First attempt: top-k pruning at layer 2, keep 128 tokens. Quality on short questions barely moved. Quality on "what's the total on line 4 of the invoice" collapsed, because line 4's tokens had been dropped before the model knew the question was about a number.

That's the failure CatchUp is built for, and the diagnosis I got from my own traces matched theirs: when I forced the dropped tokens back in, they came back wrong. The hidden states were unchanged from the moment they were parked, so the model was grounding an answer in a representation from eight layers ago. The router wasn't broken. The cache of deferred tokens was.

I haven't run Xiaomi-Lens-0 end to end, so treat my read on it as architectural, not empirical. What I did test was the [SEG]-at-the-end pattern on satellite tiles, using a standard embedding-as-mask setup. The failure mode GeoReason describes is exactly what happens. The model produces a fluent, correct description of a harbor, then drops the mask on the wrong pier. Move the segmentation condition into the query and the same model gets it right more often, without any retraining. I didn't need their benchmark to believe the mechanism.

Common Pitfalls

Optimizing the token count instead of the clock. At 64 tokens CatchUp cuts active visual tokens by 88.9% and latency by 8.0%. If your rollout plan assumed savings scale with token reduction, you'll ship a change nobody can measure in p99.

Re-admitting deferred tokens with stale hidden states. This is the trap inside recoverable routing. A token that skipped ten decoder layers needs its representation brought forward, not restored. If grounding got worse after you added re-admission, the router is fine and the re-admission path is the bug.

Assuming the top-k visual set is stable across depth. It isn't. A patch ranked 200th at layer 4 can rank 3rd at layer 20 once the query semantics settle. Pruning once and freezing the decision is the mistake; re-scoring per interval is the fix.

Giving a diffusion cache a uniform refresh budget. Repetition clusters at the positions about to be committed. Spend the refresh budget there, the way Decode-Aware Refresh does, instead of spreading it evenly across the sequence.

Emitting the segmentation token at the end of the reasoning chain. By token 400 the [SEG] embedding is conditioned on a paragraph of text, not on pixels. Put it in the query and let it anchor the whole generation.

One Thing to Remember

The efficiency number that gets published is almost never the number you feel. 88.9% fewer tokens, 8.0% faster. 99.6% down to 17.0% repetition, under 15% overhead. Read every paper in this space by finding the second number first.

The Bottom Line

If you're serving a vision model at scale and your p99 is dominated by prefill, start with depth-aware token re-admission rather than one-shot pruning. You get 92.17% of full-model quality at 64 tokens across six benchmarks, and because it's training-free you can A/B it in a day. Budget for roughly 8% latency improvement, not 80%.

If you're running a diffusion-based MLLM with an approximate cache and your long responses repeat, CoTA++ is the cheapest fix on the table: training-free, under 15% overhead, and it takes the worst-case repetition rate from 99.6% to 17.0%. Pair it with a retry heuristic, because 17% is not zero.

If you need fine-grained region output, stop building two-tower pipelines. Xiaomi-Lens-0 gets masks from the language model's own latents at 4B parameters, which fits on a 24GB card, and beats its Qwen3.5 base by 5.6 points across eight RefCOCO splits. Copy the architecture now and wait on the weights.

Sources

• CatchUp: Depth-Aware Re-Admission of Visual Tokens for Efficient Large Vision-Language Models. arXiv:2610.23324, code at github.com/pej0918/CatchUp.
• CoTA++: Understanding the Repeat Curse in dMLLMs from an Information Flow Perspective. arXiv:2610.23290.
• Xiaomi-Lens-0 Technical Report. arXiv:2610.23279, project page at seerray-lab.github.io/Xiaomi-Lens-0.
• GeoReason: Bridging Logical Reasoning and Spatial Fidelity in Remote Sensing Segmentation. arXiv:2610.23416, code at github.com/Fanc0126/GeoReason.