The adaptation layer moved, not the models

Four projects crossed my feed this week. They don't directly compete. They're all pulling the same lever: make model adaptation smaller, cheaper, and more modular than a full fine-tune.

LARA bolts low-rank residual adapters onto a frozen LLM and routes between behaviors at inference. Laya is a 421M-parameter decision model that scores [MASK] markers instead of generating tokens. Agent Lightning is Microsoft's 3,500-line RL framework for training agents through their real tool harnesses. Unsloth keeps pushing down the cost of running any of it. Even Marin, the pretraining-from-scratch program, fits the pattern: it publishes every failed experiment because base-model building is the one cost left that nobody has made modular yet.

The unit of adaptation has shrunk. Nobody here is publishing another LoRA of a 70B model. The new default is a small add-on that changes what a frozen model does without retraining the whole thing.

Key numbers

• 421M parameters: Laya's entire model. Inference needs no GPU.
• 35 ms: one Laya forward pass. Fast enough for token-level routing in a live pipeline.
• 6K training samples: what lifted a 9B coding agent from 41.8% to 56.4% on SWE-bench Verified.
• 2x faster, 70% less VRAM: Unsloth's baseline fine-tuning claim. An 8B fine-tune on a 24GB consumer card is now routine.

LARA: behaviors as files

The LARA repo is a research project, but the demo lands immediately. You train a low-rank residual adapter at selected layers and leave the base model's weights alone. The resulting behaviors are small enough to keep separately, and you can load, remove, blend, or route them at inference time.

The Mixture of Behaviors demo is the part that stops feeling academic. Several independently trained behaviors share one frozen model, and a soft router picks or combines them token by token. One backbone handles coding, math, medical, and summarization behaviors instead of four separately adapted checkpoints. When I ran the style demo (Hemingway, Fitzgerald, Gertrude Stein), switching writers was a router lookup, not a model reload.

The repo includes a direct comparison with LoRA, and the distinction matters. LoRA modifies weights and merges into the backbone. LARA keeps the adaptation additive and residual, which is what makes behaviors composable and swappable at runtime instead of baked in. The project is still in active research, but the library is usable now, with training code, examples, and paper reproduction instructions.

Laya: don't generate, score

JEV, the architecture that started this thread, got a mystery name and a lot of squinting. The community read on the diagram was blunt: take a frozen LLM, skip the text generation, and read the confidence scores from the matrix before decoding.

What the community is saying: I remember scrolling the JEV thread and watching people try to reverse-engineer the same vague diagram. After enough back-and-forth, the consensus take was simple: skip the generation step and wire the pre-decoding scores into a decision. That reading makes it a router: a scoring function over options, not a generator. The method was fine. The presentation did it no favors.

Laya is the open answer to JEV. It pairs a ModernBERT-large encoder with a scratch Transformer head that scores [MASK] option markers, resolving typed schemas in a single ~35 ms forward pass. The whole thing is 421M parameters, trained on one RTX 6000 Pro with 96GB of VRAM, and you can run it on a low-end PC. Those two numbers carry the practical weight: a decision model small enough to sit in front of a big model and route traffic without a GPU of its own. There's a live HF space if you want to poke it.

The training stack is where Laya differs from a typical head fine-tune. Instead of plain cross-entropy on final answers, the author used RLCD, an unofficial policy-gradient approach that optimizes against strictly proper scoring rules. Maximum reward is only achievable when the model outputs true, mathematically calibrated probabilities. For routing decisions, that distinction is everything: a confident-but-wrong router is worse than a slow generator.

The dataset is 100% human-annotated, 25,000+ examples across intent routing, fact-checking, moderation consensus, prompt guardrails, rubric scoring, and multi-turn conversation trajectories. No synthetic shortcuts. That's a small corpus by pretraining standards, but for a 421M model that only has to score options, it's enough to be useful.

Quick Take: The unit of adaptation is shrinking. Residual adapters, decision heads, and RL loops trained on a few thousand examples are replacing the full fine-tune as the default move, and they run on hardware that used to be a joke for ML.

Agent Lightning: RL with the harness in the loop

Agent Lightning is the most directly useful release of the four. It's a ~3,500-line framework for agentic RL that trains agents through their real harnesses: tools, context, control flow, and environment stay in the loop, and the agent code changes zero. v1.0 is a full refactor built around three components.

The Trainer runs verl and vLLM, builds training samples, and updates the policy. The API Gateway proxies model requests and captures the traffic as training data. The Rollout Controller runs agents locally or as Kubernetes Jobs. Because agents talk to the model through an OpenAI-compatible proxy, you train the exact agent you'll deploy, no simplified gym environment.

The headline result: 6K training samples took a Qwen3.5-9B workflow from 41.8% to 56.4% on SWE-bench Verified, a gain of 14.6 percentage points. That's the difference between a coding assistant you babysit and one you can hand a repository. The repo releases the full pipeline, including data cleaning and reward-hacking prevention.

Two details in the changelog matter more than they look. First, the gateway returns token IDs through the OpenAI-compatible API instead of text, which kills the retokenization drift that silently corrupts long agent trajectories. Second, trajectory-level aggregation made training faster by collapsing rollouts before they hit the trainer. Small fixes, but they're the kind of thing that separates a framework that works on paper from one that converges on a cluster.

The original paper covered the zero-code-change trick; v1.0's technical report covers the harnessed setup, and the repo now runs agents as native Kubernetes Jobs without external sandbox services. For anyone doing agent RL on actual products, this removes the last excuse to train against a toy environment.

Unsloth: the efficiency floor keeps dropping

Everything above assumes you can train cheaply. Unsloth is the layer that makes that assumption true. The library's claims have been stable for a while now: 2x faster training, 70% less VRAM, no accuracy loss. The interesting part is how far that extends past vanilla fine-tuning.

Workload	Speedup	Memory saved
gpt-oss (20B) fine-tune	2x	70% less
gpt-oss (20B) GRPO	2x	80% less
Qwen3.5 (4B)	1.5x	60% less
Llama 3.1 (8B) Alpaca	2x	70% less
Orpheus-TTS (3B)	1.5x	50% less
MoE training (DeepSeek, GLM, Qwen, gpt-oss)	12x	35% less

The MoE row is the one to watch. Training sparse models 12x faster with 35% less VRAM changes the economics of experimenting with expert routing, which matters for anyone following the LARA-style modular path. The RL side is covered too: GRPO, DPO, and FP8 training are supported, and the new batching algorithms claim 7x longer context windows for RL runs. The desktop app and the unsloth start claude and unsloth start codex commands also mean the line between "train a model" and "point an agent at a local model" is gone.

Not every number reproduces on every setup. The 70% VRAM figure has held up in my own fine-tunes; the 2x throughput depends on kernel support for your exact model, which the repo documents per model.

The full-stack counterpoint: Marin

Marin is the exception that clarifies the trend. It's a research program and platform for building foundation models, and its current focus is pretraining a large MoE model from scratch, around 5e24 model-FLOPs and 500B+ total parameters. Its Delphi suite scales a training recipe from 3e18 to 1e23 FLOPs, with scaling laws that extrapolate 300x past the fit.

It belongs here for one reason: every modular approach above inherits its ceiling from the frozen backbone. Marin's 8B model already beat Llama 3.1 8B on the team's base-model benchmark suite, and it publishes checkpoints, mixture pipelines, and failed experiments as it goes. If the adaptation layer is getting modular and cheap, the base-model layer is where the remaining cost and leverage sit.

Project	Unit of adaptation	Training budget	Inference footprint	What you get
LARA	residual adapters on a frozen LLM	one GPU per behavior	one backbone + router	N behaviors, one model
Laya	[MASK]-scoring head on ModernBERT	one 96GB GPU, 25K examples	421M params, ~35 ms	calibrated routing on CPU
Agent Lightning	agent policy via proxy	6K samples for +14.6 SWE-bench	model unchanged	RL trained on real harnesses
Unsloth	training and serving tooling	consumer GPU	n/a	2x faster, 70% less VRAM
Marin	pretraining from scratch	5e24 FLOPs MoE	huge	open checkpoints and process

Common Pitfalls

These projects all have sharp edges. Here's what trips people up.

1. Routing gets expensive past a few behaviors. LARA-style setups replace four checkpoints with one backbone plus adapters, which is a big memory win. Push to 20 behaviors and router state and adapter storage start eating the savings. Keep behaviors low-rank and prune what you don't serve.

2. Decision heads live and die by calibration. Laya's RLCD training exists because a head that outputs confident junk is worse than a model that generates slowly. Train a [MASK]-scoring head with plain cross-entropy, and don't trust the probabilities as routing weights until you've recalibrated them on held-out data.

3. Agentic RL without reward-hacking defenses rewards the harness, not the task. Agent Lightning ships reward-hacking prevention because agents will learn plausible tool calls and fabricated traces that game the reward. Skip that step and your SWE-bench gain becomes an illusion: the agent passes the reward model and fails the repo.

4. Unsloth's memory claims assume you follow the precision path. The library routes you to bf16 and specific kernel versions for a reason. Copy the training loop into your own script, force fp16, and you'll hit silent overflow in attention; the "no accuracy loss" guarantee evaporates. Use the library presets, and export GGUF from the same quantization family you train in, or the export shifts behavior.

5. Small curated corpora overfit benchmarks fast. 25K annotated examples and 6K training samples are small by design, which means contamination and eval leakage hurt more. Track per-seed variance and hold out a slice of the real distribution, not just the benchmark split.

One thing to remember

None of these tools fix a weak base model. LARA routes around a frozen backbone, Laya stands on ModernBERT, Agent Lightning tunes a policy that inherits everything its base model knows. The adaptation layer is getting cheaper and more modular by the week, but the ceiling is whatever you froze in the first place. Pick the strongest base model you can actually serve, then bolt on the smallest thing that fixes your problem.

The Bottom Line

• If you ship one model into many use cases, adopt LARA-style composable adapters. You get coding, math, medical, and summarization behaviors for the memory cost of one backbone plus a router, and swapping behaviors becomes an inference-time lookup instead of a redeploy.
• If you need ultra-low-latency decisions (moderation, guardrails, intent routing) and can't afford token generation, use a Laya-style non-autoregressive head. 421M parameters and a 35 ms forward pass run on CPU, and calibrated probabilities beat a confident argmax every time.
• If you're building agents that use real tools, skip the toy environments and train with Agent Lightning. The proxy-based harness setup plus reward-hacking defenses is what turned 6K samples into a 14.6-point SWE-bench jump. Watch Unsloth's MoE training numbers: 12x faster sparse-model training will reshape the cost of all of this within two release cycles.