The shared failure mode

Robot world models keep breaking at the exact moment something touches the table. You train on a good dataset, the frames render, the rollout looks smooth. Then a gripper misses a cup by three centimeters, and the model cheerfully predicts a successful grasp instead of showing the cup wobble and fall. That gap between "plausible video" and "grounded prediction" is the core reason world models underdeliver in robotics.

Three papers from this month's arXiv crop attack the problem from three angles. DreamTrue makes action-conditioned video prediction actually follow the commanded action, using counterfactual post-training to expose the model to failures. LeWAM removes pixel reconstruction from world action models entirely, building a decoder-free JEPA latent that predicts forward, backward, and inverse dynamics. WOVEN reframes the whole thing as a training primitive for multimodal LLMs: visual transition reasoning, packaged as 36,076 examples that improve 22 of 26 external benchmarks after fine-tuning.

Start with the failure mode that hurts most in practice: action following.

DreamTrue: train the model on its own failures

DreamTrue is a multi-view, cross-embodiment robot world model for action-faithful and physically plausible video prediction. The team hits two obstacles when training on existing robot datasets. First, imprecise calibration: the mapping between the commanded action trajectory and where it lands in pixel space is slightly off, embodiment by embodiment. Second, coverage: most robot datasets record successful interactions, so the model never learns what a failed grasp, a slip, or a slow collision looks like, and it biases every prediction toward success.

The calibration fix is clever. Instead of feeding raw action vectors as conditioning tokens and hoping the model figures out the mapping, DreamTrue renders action trajectories into image-space conditions. Then an offline geometric calibration step aligns those conditions with the target videos. The action conditioning ends up living in the same coordinate frame as the pixels, which is what makes cross-embodiment training work at all.

The coverage fix is where the paper gets interesting. After initial training, DreamTrue modifies recorded action trajectories and generates future videos under a wider range of actions and contact configurations. This is counterfactual post-training: the model sees what happens when the gripper closes 200ms early, when force lands at the wrong angle, when the object slips. There's no paired ground-truth future for these predictions, so the authors built a human-annotated video dataset covering robot, object, and interaction defects and trained an embodied video reward model on it. That reward model's scores guide reinforcement-learning post-training toward physically plausible outcomes.

The numbers are hard to argue with. On AgiBot, DreamTrue posts the best action-following scores reported so far, and the human-assessed interaction defect rate drops from 48.12% to 6.25%. Roughly an 8x reduction in visible physics failures, measured by humans, not by FVD.

One thing I keep coming back to: the reward model is a separate artifact that does the heavy lifting. If you want to reproduce this, budget for the annotation effort. The pipeline is data work plus RL post-training, not architecture tricks.

LeWAM: drop the decoder, keep the dynamics

The LeWAM authors start from a different complaint. World action models predict actions and future observations, and most of them do it from a reconstruction-based representation. Reconstruction carries noisy, redundant information: lighting changes, texture detail, background motion. All of that noise complicates the downstream dynamics prediction, even though the state you actually care about, "where is the robot and where is the object", is low-dimensional.

LeWAM is a bidirectional transformer for forward, backward, and inverse dynamics plus policy prediction, trained end-to-end on a decoder-free JEPA latent. No pixel reconstruction head. The latent is learned and shared across all four modes: forward prediction, backward prediction, inverse dynamics from observations to action, and direct policy.

Three results matter here. Linear probes read robot and object state from LeWAM's latent better than from a forward-only JEPA world model, and far better than from a reconstruction-based WAM, while ignoring visual distractors as well as the forward-only model. Closed-loop evaluations match a regular flow-matching policy trained on the same encoder at matched size, except LeWAM also gives you a world model for free. And the planning result is the most counterintuitive: when you plan with a world action model, sampling raw actions lets dynamics-model inaccuracies compound, so planning in the noise space of the policy head actually improves closed-loop performance.

That last point deserves a pause. Most MPC pipelines sample actions directly and roll them out through the dynamics model. LeWAM shows that if your policy head is a learned flow, you're better off sampling perturbations in that flow's latent noise space. The dynamics model still guides the search, but it doesn't get to corrupt the action distribution.

Quick Take: All three papers converge on the same bet: the bottleneck in embodied AI right now is the supervision signal, not the backbone. World-model training works when it targets the transition itself, not the pixels.

WOVEN: transition reasoning as a shared training primitive

WOVEN comes at this from the multimodal LLM side, and its diagnosis is the most sweeping. MLLMs are bad at spatial, embodied, physical, and temporal reasoning. The paper's hypothesis is that these failures share a root cause: a deficit in visual transition reasoning, the ability to predict what changes when objects and actions interact. Existing benchmarks document each deficit in isolation but don't allow controlled comparison across scenes, actions, and reasoning operations.

So WOVEN builds both a training source and a benchmark: 36,076 examples of visual transitions, organized by 20 scene types, 5 action types, and 8 reasoning types, generated from video-pretrained generative models. That last part matters: the supervision is synthetic but visually realistic, and it's structured by the reasoning operation being taught.

The evaluation of 38 frontier MLLMs, including GPT-5.4 and Qwen3-VL-235B-A22B, is bleak. The 235B-parameter Qwen variant (22B active) still falls far below human performance on transition reasoning, and the failures recur across model families and persist with scale. Scale alone doesn't buy you physics. But fine-tuning on WOVEN does: training subsets of about 2,000 items each improved 22 of 26 external benchmarks, by up to 27.3 percentage points. And WOVEN data can replace 30-50% of a task's own training data with comparable accuracy.

2,000 examples is a weekend labeling effort, or a single batch job against a generative model. That's what makes this result usable: you don't need to collect robot data to teach an MLLM to reason about transitions, you need structured transitions, and the structure matters more than the source domain.

Key numbers

• 48.12% → 6.25%: DreamTrue's human-assessed interaction defect rate on AgiBot after counterfactual post-training.
• 36,076: WOVEN transition examples across 20 scenes, 5 action types, and 8 reasoning types.
• 22 of 26: external benchmarks that improved after WOVEN fine-tuning, with gains up to 27.3 percentage points.
• 30-50%: share of a task's own training data that WOVEN transitions can replace.

Three pipelines, one pattern

Put the three papers side by side and a shared structure appears. Each one treats the world model as a supervision problem rather than a modeling problem. DreamTrue re-engineers the training distribution so failures are visible. LeWAM re-engineers the representation so dynamics are learnable. WOVEN re-engineers the supervision taxonomy so the reasoning operation is the unit of data selection.

And the differences matter, because they map to different workloads.

	DreamTrue	LeWAM	WOVEN
What it predicts	future video frames	actions plus latent future states	visual transitions, framed as reasoning tasks
Main fix	action calibration + failure coverage	noisy reconstruction representations	missing transition-reasoning supervision
Training signal	counterfactual RL post-training with a learned reward model	four dynamics modes on a decoder-free JEPA latent	36,076 structured generative rollouts
Best result	first place, AgiBot World Challenge 2026 world model track	matches a flow-matching policy while adding planning	22 of 26 benchmarks improved, up to 27.3 points
What it costs you	a human-annotated defect video set for the reward model	four training modes and latent probes for eval	curated 2,000-item subsets per reasoning type

Common pitfalls

Pitfall 1: Optimize for pixel error, not action faithfulness

If you evaluate a world model with FVD or SSIM, you can ship a model whose frames look clean but whose reactions to the commanded action are wrong. DreamTrue's baseline had a 48.12% defect rate, and nothing in the reconstruction metrics caught it. Add action-conditioned probes to your eval, or better, a human-rated defect taxonomy for robot, object, and interaction failures.

Pitfall 2: Assume action trajectories transfer across embodiments

Camera intrinsics and extrinsics differ between robot arms, and raw action vectors don't align with pixels across platforms. If you condition on actions directly, cross-embodiment training degrades quietly. Render the action trajectory into image space and run offline geometric calibration first, the way DreamTrue does.

Pitfall 3: Train only on successful rollouts

Robot datasets are filtered for success, so the model never learns failure transitions. It will predict successful grasps even when the physics says otherwise. Perturb recorded actions and contact configurations, generate the counterfactual futures, and include them in training. The defect rate drop from 48.12% to 6.25% is mostly this.

Pitfall 4: Plan in raw action space

When you use a world action model for MPC, sampling raw actions amplifies the dynamics model's inaccuracies. LeWAM's noise-space planning, perturbing the policy head's latent instead, improved closed-loop performance in their evaluations. If your MPC rollouts look brittle, change the sampling space before you change the model.

Pitfall 5: Curate world-model data by scene instead of reasoning operation

WOVEN's controlled comparisons are blunt about this: select supervision by the reasoning operation it teaches, not by the actions, scenes, or domains it shows, and prefer larger changes to the visual state. Domain-matched data that teaches the wrong operation transfers worse than mismatched data that teaches the right one.

One thing to remember

World modeling is becoming a supervision problem. DreamTrue rebuilt the training distribution, LeWAM rebuilt the representation, and WOVEN rebuilt the benchmark taxonomy. If your world model is stuck, ask which of those three is broken for you: the data distribution, the latent, or the reasoning signal. The architecture arms race is over; the data design race is just starting.

The Bottom Line

If you're building an action-conditioned video predictor for real robot data, adopt DreamTrue's recipe: render action trajectories into image space, calibrate them geometrically, and post-train on counterfactual failures scored by a reward model. That combination alone cut the human-assessed defect rate by roughly 8x.

If you want a world model for planning and control rather than video generation, skip pixel reconstruction and follow LeWAM: train forward, backward, and inverse dynamics on a decoder-free JEPA latent, and plan in the policy head's noise space, not raw action space.

If you're fine-tuning an MLLM that needs to reason about physics, feed it WOVEN-style transition supervision and select examples by reasoning operation. About 2,000 examples per reasoning type carry most of the benefit, and you can replace up to half of a task's own training data. One thing to watch: the reward-model and transition-benchmark threads are converging, so expect learned physical-plausibility scoring to become a standard component of world-model training within the next year.