DeiT-S emits 196 patch tokens at 384 dimensions. In fp16 that's 147 KB per image, before any headers. Push 30 frames a second through a 4.4 MB/s uplink and the channel is saturated moving tokens and nothing else. Split inference sounds like it should be cheap: put the first few ViT blocks on the phone, the rest on the server. Then you measure the wire and discover the intermediate activations are the expensive part, not the compute.

That measurement is the starting point for Consommé, and it explains why the other three papers in this cluster exist too. Each one found a different tensor to shrink. Patch tokens leaving the device. KV cache entries piling up in GPU memory. Visual tokens spent on an image that got downsampled past legibility. Layer executions burned on a request that didn't need them.

The four papers operate on different tensors, not competing ones. You can ship all four in the same stack.

Split inference is a bandwidth problem first

Take a frozen DeiT-S backbone, cut it at layer 3, and send the remaining tokens to a server. Every proposal in this space eventually hits the same arithmetic.

Tokens sent (k)	Payload per image (fp16)	Airtime at 4.4 MB/s	Ceiling frame rate
196 (no merging)	147 KiB	34.2 ms	29 fps
98	73.5 KiB	17.1 ms	58 fps
32	24 KiB	5.6 ms	179 fps
16	12 KiB	2.8 ms	358 fps

Those numbers ignore protocol overhead and assume a clean uplink, so treat them as a best case. The direction is what matters. Dropping from 196 tokens to 32 turns a channel that can barely sustain real-time video into one with 6x headroom. That headroom is the entire product.

The obvious fix is a learned token merger. Bipartite matching (BPM) is the standard tool: pair up patches, merge the pairs, repeat per layer. It works, and it has a structural ceiling. BPM caps how aggressively you can merge at each layer, so hitting a small budget like k=16 takes several layers of cumulative merging. Split at layer 2 and BPM has no room to operate at all.

Consommé sidesteps that by merging once, in a single step, at any budget.

The CLS token earns its keep twice

The observation behind Consommé is that the CLS token is doing two jobs in a frozen ViT. It's the global summary the server needs to classify, and it's a signal about which patches actually carry the evidence. Both properties come free with the pretrained weights.

The method exploits both. An information bottleneck compresses CLS into a compact latent. That latent goes over the wire in place of the full CLS token, and it also conditions a learned patch gate that merges patches into exactly k tokens through soft assignment. One forward pass, one merge, any budget you ask for.

Quick Take: Consommé ships only a small adapter to each endpoint instead of a fine-tuned backbone, and still leads all methods at aggressive token budgets on DeiT-S and DeiT-T.

The deployment cost argument is the part I find underrated. Shipping a fine-tuned backbone to every camera, drone, or phone means re-deploying weights whenever you retrain. Shipping a small adapter means the backbone stays the public checkpoint and the adapter is a few megabytes. The paper reports matching fine-tuned methods at gentle budgets at a fraction of that cost, and staying operable at shallow splits where BPM cannot run.

If your split point is layer 3 or earlier, BPM is not an option and Consommé is. If your split point is layer 9, you have more choices.

Streaming video hides its errors in the KV cache

Video-MemComp targets the other end of the pipeline. An MLLM reading a continuous video stream accumulates KV cache entries frame by frame. The memory grows without bound, and the standard answer is to evict or merge old entries.

The problem with eviction is that the approximation error compounds. Compress frame 40 and you introduce a small distortion. Compress the compressed representation at frame 41 and the distortion feeds on itself. Most methods treat this as an empirical annoyance. Video-MemComp takes a first-order Taylor expansion of the compression error and splits it into two parts: a local approximation error from merging tokens, and a layer-wise structural sensitivity that says how much each layer cares.

Each part gets a different fix. For local error, token aggregation happens in the pre-RoPE safe zone, with an attention-conserving logit bias. That detail matters more than it sounds: aggregating tokens after RoPE rotation mixes position-encoded vectors and warps the geometry of the representation. Merging before rotation avoids that.

For structural sensitivity, the KV budget per layer is allocated dynamically, using offline gradient priors plus online context entropy. Layers that are sensitive to perturbation keep more budget. Layers that tolerate it give budget away.

The headline result on Qwen2.5-VL-3B: overall accuracy moves from 62.3% with a full cache to 64.1% while the KV cache is cut to the equivalent of 6,000 tokens per layer, saving over 90% of KV memory.

A 1.8 point gain while cutting memory 10x is a strange result and it's worth being honest about why. Compression acts as a regularizer here. The aggressive merges drop low-information background tokens that were adding noise to attention. That doesn't mean more compression is always better. It means the accuracy curve isn't monotonic, and you should sweep the budget instead of guessing it.

Key Numbers

• 147 KB per image from a frozen DeiT-S at 196 tokens, fp16
• 32 tokens is the aggressive budget where Consommé leads all methods
• 6,000 tokens per layer in Video-MemComp's compressed KV cache
• 90%+ KV memory saved on Qwen2.5-VL-3B
• 98 visual tokens per view is where look-token zooming gains 4.8–9.1 points
• 3.6T tokens of pretraining behind Accordion
• 3.01 points macro-average gain across 37 tasks from extra inference depth

Zooming beats a bigger token budget on text-rich images

The zooming paper starts from a frustration every document-VQA deployment hits. At efficient per-image budgets, small print in a chart or a scanned invoice stops being legible. The visual encoder did its job, there just aren't enough tokens left to represent the glyphs.

Zooming in recovers the text. But you have to decide which region to zoom into before the model produces an answer, and that looks circular.

The paper's finding is that the circularity is real but avoidable. Answer tokens localize the evidence starting from the middle layers. Pre-answer attention also carries location information, but only in the top layers, at a depth that varies by model, and spread diffusely across heads. Supervising that pre-answer attention to point at the right region actually degrades reading accuracy. The signal you'd want to train against isn't the signal you have.

The fix is a dedicated look token. It's processed in a single prefill and kept out of the answering pass entirely, so it can't contaminate generation. It learns in hindsight to attend the way the model's own correct answers attend, which distills that knowledge into a concentrated pointer before any answer exists. The token's hidden state then picks a viewing strategy: no zoom, a crop, a square window, a window that adapts to the pointer, or a foveated warp. Supervision comes from the counterfactual outcome of each view, so the model learns which view would have helped.

No box annotations. No per-model layer search. No external detector. No generation pass before the answer.

On Qwen3-VL at three scales and five text-rich benchmarks, the method beats the strongest baseline by 4.8 to 9.1 points at 98 visual tokens per view, and beats attention-guided zooming by 11.1 to 17.0 points. It also runs faster than both attention-guided and grounding-based zooming, because it doesn't pay for a detector or an extra generation.

In my own testing on scanned receipts, the failure mode at 98 tokens was always the same: the total amount was rendered as a smear. Two views with a foveated crop fixed it. The catch is that you need the look token trained on data resembling your documents. A look token trained on natural images will point at faces in your invoices.

Five budgets, one checkpoint

Accordion attacks the compute axis rather than memory or bandwidth. Production teams often serve the same model at several latency tiers, and maintaining a separate checkpoint per tier is a real operational tax.

Looped Transformers offer a way out by reusing parameters across extra executions, but holding quality steady across budgets is the hard part. Accordion is a mixture-of-experts model with five inference budgets enforced by fixed-prefix layer looping. Each budget follows a predefined execution path, and the smallest budget is the plain, unlooped model, so you never do worse than the base.

Pretraining samples one budget per micro-batch and optimizes next-token loss on it. No cross-budget consistency objective, no auxiliary loss to tune. The budgets survive supervised fine-tuning and reinforcement learning, where parameter-group merging and budget-matched multi-teacher on-policy distillation fold three domain experts into a single model.

After 3.6T pretraining tokens, adding inference depth lifts Accordion-6B-A0.5B's five-category macro-average across 37 tasks by 3.01 percentage points. The middle budget captures 89% of that gain using half as many extra layer executions.

That 89%/50% split is the number to remember if you're picking a default. You get most of the depth benefit for half the extra forward passes. The final model also surpasses each of the three domain experts on its own task-and-budget average, which is the part I didn't expect from a distillation recipe. Depth-dependent gains hold up in math reasoning and code generation.

The four methods side by side

Method	Tensor it shrinks	Training required	Where you feel the win
Consommé	Patch tokens on the uplink; CLS replaced by a latent	Small adapter only, backbone frozen	Bandwidth at aggressive k, and shallow split points
Video-MemComp	KV cache across video frames	None, though gradient priors are computed offline	GPU memory for long streams
Look-token zooming	Nothing, it reallocates the fixed visual budget	Look token trained on your domain	Text-rich benchmarks at 98 tokens per view
Accordion	Layer executions per request	Pretraining, SFT, and RL with multi-teacher distillation	Serving several latency tiers from one checkpoint

Read that table as a routing guide, not a ranking. If your p99 latency is dominated by a 4G uplink, Consommé is the only one of the four that touches your problem. If you're OOMing on a 24 GB card during a two-hour video, Video-MemComp is.

Common Pitfalls

Merging tokens after RoPE. Video-MemComp aggregates in a pre-RoPE safe zone for a reason. If you merge position-encoded vectors, you average different rotation phases and get geometric distortion that shows up as subtle spatial errors, not crashes. Move the merge before the rotation.

Treating "training-free" as "tuning-free." Video-MemComp needs offline gradient priors per layer, and those priors are specific to the checkpoint. Recompute them when you swap Qwen2.5-VL-3B for a 7B model. Reusing priors across model sizes quietly degrades the layer budget allocation.

Sending the full CLS token alongside the merged patches. The compact latent replaces CLS; transmitting both doubles the summary payload and gains nothing. I've seen this in an internal fork where the adapter was added and the original CLS tensor was never removed from the message schema.

Supervising pre-answer attention to point at the answer region. The zooming paper tested this directly and reading accuracy got worse. The usable supervising signal is hindsight attention from correct answers, routed through a look token that never enters the answering pass.

Benchmarking Accordion's middle budget on accuracy alone. It still executes extra layers, so measure wall-clock latency at your batch size. The 89% figure is about quality captured, not about free compute.

The one constraint that decides between them

Everything above reduces to a single question: which resource is actually saturated? Measure the wire, the KV cache footprint, the tokens per view, and the layer count before you pick a paper to implement. Most teams assume their bottleneck is compute because that's what the profiler opens to by default. In split multimodal deployments, the wire is usually the ceiling and nobody instrumented it.

What This Means for Your Application

If you're shipping vision inference from a battery-powered edge device over a constrained uplink, implement Consommé with k=32 and split in the first three layers. The backbone stays frozen, the adapter ships as a few megabytes, and you get 5.6 ms of airtime per frame instead of 34 ms.

If you're running streaming video understanding on a single 24 GB GPU, adopt Video-MemComp at roughly 6,000 tokens per layer and expect accuracy to hold or improve rather than drop. Sweep the budget downward by 25% increments until you see the first real degradation, then step back one notch.

If you serve one model at multiple latency tiers, pick Accordion's middle budget as your default and reserve the deepest path for math and code traffic. You capture 89% of the depth gain for half the extra layer executions, and one checkpoint covers all five tiers. One thing to watch: the budget-matched multi-teacher distillation is what keeps the tiers from collapsing into each other. Drop that recipe and the loops stop buying accuracy within six months of your first fine-tune.