The Full Loop: World Models That Act on What They Don't Know

W33 Trend Tutorial | Difficulty: Advanced | Research area: Probabilistic world models

Closing the Loop

This arc has been building one argument in four moves. We reframed attention as kernel regression and argued that scale abandoned a tool it still needs: calibrated uncertainty. We watched a point-estimate 3D scene hallucinate floaters into space no camera ever constrained, and fixed it with a distribution over geometry. Last week we made calibration the hinge of neuro-symbolic reasoning — the thing that lets a fluent perceptron hand predicates to a rigid reasoner without the chain snapping on the first misread pixel.

Each week ended on the same note, and each time we deferred the same question. A calibrated confidence number is a permission slip. Permission to do what?

This week: to act. And acting is where the arc's thesis gets its hardest test — because the thing we spent three weeks learning to calibrate is a single-step prediction, and planning needs a rollout.

The Classical Answer: Plan Through the Posterior

The oldest correct answer to "what do you do with uncertainty" is fifteen years old and still the cleanest demonstration in the literature. PILCO (Deisenroth & Rasmussen, ICML 2011) learns a Gaussian-process model of a robot's dynamics and — this is the whole trick — plans by propagating the model's posterior forward through time (via moment matching, under a Gaussian approximation) rather than a maximum-likelihood trajectory. Every rollout carries its own doubt, so the policy optimizer is penalized for plans that depend on regions the model has never seen.

The payoff was not marginal. PILCO solved cart-pole swing-up-plus-balance on real hardware with 17.5 seconds of total interaction with the physical system, and reports outperforming "any other algorithm by at least one order of magnitude" in data efficiency — a comparison against a decade of prior work that had learned the task from scratch, some of it on balancing alone. The lesson generalizes past GPs: a model that knows what it doesn't know does not merely fail more gracefully, it learns faster, because it stops wasting trials confirming things it already knows.

PETS (Chua et al., NIPS 2018) carried the idea into deep networks and, in the process, gave the field its working vocabulary. It separates aleatoric uncertainty — the irreducible noise in the system, captured by having the network output a distribution's parameters — from epistemic uncertainty, the subjective doubt from limited data, captured by an ensemble and expected to vanish in the infinite-data limit. Planning with both, PETS reached PPO's asymptotic performance in under 100 trials, using (per its abstract) 8× fewer samples than Soft Actor-Critic and 125× fewer than PPO on half-cheetah.

Then the field scaled, and something quietly dropped out. DreamerV3 beat specialized methods on over 150 tasks with a single configuration and became the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula — a result significant enough to land in Nature in April 2025. TD-MPC2 trained one 317M-parameter agent to perform 80 tasks across different embodiments and action spaces, planning in the latent space of a decoder-free world model. Both are magnificent. Neither makes epistemic uncertainty a first-class citizen of the planning objective the way PILCO did. Generality arrived; the posterior got left behind.

The Result That Breaks Our Thesis

Here is where an honest arc has to turn on itself.

The natural fix is obvious: bolt the ensemble back on. Estimate epistemic uncertainty over the learned latent dynamics, penalize plans that wander into disagreement, and you should recover PILCO's discipline at Dreamer's scale. Serious work does exactly this — RWM-U with MOPO-PPO (Li, Krause & Hutter) extends autoregressive world models with epistemic uncertainty estimation propagated across long horizons, and reports beating uncertainty-unaware baselines on real quadrupeds and humanoids.

But two 2026 results say the obvious fix is not reliably a fix at all.

The first is Biased Dreams (Berger et al., RLC 2026), and it is a genuinely uncomfortable finding. Studying recurrent state-space models — Dreamer's substrate — the authors find that although ensemble disagreement captures local epistemic uncertainty, it does not reliably reflect the global compounding model error accumulated over long latent rollouts. Worse, they document an attractor behavior: rollouts get pulled toward well-supported regions of latent space, where uncertainty shrinks even as the discrepancy from true dynamics grows. The model becomes more confident precisely as it becomes more wrong, because it has drifted somewhere it has seen a lot of data — just not the somewhere the robot is actually in.

Read that against W28. We spent a week arguing that calibration is what makes confidence meaningful. Biased Dreams is the fine print: calibrating a one-step predictor does not calibrate a hundred-step imagination. Error accumulates over the horizon while the uncertainty estimate is only ever asked a local, one-step question — and the attractor result shows the two can move in opposite directions.

The second result is blunter. A single-author ICML 2026 workshop paper, Learning from World Feedback, reports that across four world-model architectures spanning a 2× range in prediction error, MPC planning performance was statistically equivalent (TOST, n = 200) — and that dynamics-based uncertainty penalties raised collision rates from 26% to 34%, with a correlation of r < 0.15 between model uncertainty and actual task risk. Replacing the proxy with task-grounded signals — minimum lidar margin, time-to-collision, and an outcome-supervised feedback model trained on prior collision labels — cut collisions to 1–14% without retraining the world model or the planner. Workshop preprints deserve hedging, but its direction matches Biased Dreams: model uncertainty is not task risk, and treating one as a proxy for the other can actively hurt.

The honest revision to our thesis is not "calibration was wrong." It is narrower and more useful: calibrated confidence about a prediction is not the same object as calibrated confidence about a plan, and the arc has been quietly conflating them.

What Actually Works: Calibrate the Bound, Not the Belief

The 2026 systems that make uncertainty pay off share a design move, and it is not "ask the model how sure it is." It is: wrap the model in a bound calibrated against held-out reality.

V-JEPA 2 (Assran et al., Meta, June 2025) sets up the substrate. Pretrained on over a million hours of internet video, its action-conditioned variant (V-JEPA 2-AC) is post-trained on fewer than 62 hours of robot video from the Droid dataset — conditioned on end-effector actions, but with no rewards, no task labels and no success annotations — and then plans zero-shot on Franka arms in labs it has never seen, hitting 80% on pick-and-place for a cup and 65% for a box. The number that matters most for practitioners is the clock: 16 seconds per action, against about 4 minutes per action for Cosmos, a video-generative world model doing the same job. Planning in a compact latent, rather than by rendering pixels, is what makes the loop close in real time.

That substrate is what conformal methods then get to work on. Pixels to Proofs (Nath et al., June 2026) trains an action-conditioned joint-embedding world model, then uses conformal prediction to produce calibrated latent error bounds and turns those bounds into robust constraint sets for a GPU-accelerated MPC scheme — plus a separately conformalized latent constraint checker enforcing probabilistic safety during closed-loop execution. Foresight (Zhang et al., June 2026) applies the same instinct to monitoring: functional conformal prediction over action-conditioned world-model latents, trained on nothing richer than final success/failure labels, with adaptively calibrated thresholds.

The distinction is worth stating precisely, because it is the practical takeaway of the week. An ensemble's disagreement is the model's opinion about its own reliability, and Biased Dreams shows that opinion drifts. A conformal bound is a distribution-free guarantee calibrated on held-out data: it covers at the rate you asked for regardless of what the model believes about itself. When you cannot trust introspection, measure the residuals instead.

That is a real improvement and not a free lunch, and the caveat is the same shape as the critique above. Standard conformal coverage is marginal and assumes exchangeability — it promises the right average rate over the calibration distribution, not the right rate in the specific regime you happen to be in. Closed-loop control strains both halves: the agent's own actions shift the distribution it then encounters, so the exchangeability assumption is not automatically satisfied and conditional coverage can be much worse than the headline number. The honest framing is that conformal methods move the trust from the model's introspection to your calibration set, which is progress exactly to the degree your calibration set resembles deployment.

This also explains the substrate choice. Video-generative world models look like the general answer, but they are poor planners. VideoPhy-2 finds the best evaluated model reaching only 22% joint semantic-and-physical adherence on its hard subset, with characteristic failures on conservation of mass and momentum. RoboWM-Bench (April 2026) makes the point operational by converting generated videos into executable action sequences and running them in physically grounded simulation; its headline finding is that visual plausibility and embodied executability are not always aligned. Meta's own IntPhys 2 results agree from the other direction: humans score at or near ceiling on it — 85–95% across the three physical-reasoning benchmarks Meta released alongside V-JEPA 2 — while current video models sit at or close to chance. A rollout that looks right is not a rollout you can act on.

The Framework Where Acting and Knowing Are One Objective

There is a body of theory that never separated the two questions. In active inference, an agent selects policies by minimizing expected free energy, and that single quantity decomposes into a pragmatic term (how well outcomes match preferences) and an epistemic term (how much the policy is expected to reveal). Getting the algebra right matters, and the cleanest open statement is Da Costa et al.: expected free energy is minimized, extrinsic value enters as the negative expected log-evidence, and the epistemic contributions enter with negative signs — splitting further into salience (information gain about hidden states) and novelty (information gain about model parameters). Secondary write-ups routinely flip these signs; the decomposition of "pragmatic plus epistemic" belongs to negative expected free energy.

The framing earns its keep by unifying things we already use. Sajid et al. show that stripping outcome preferences out of expected free energy reduces active inference to optimal Bayesian experimental design — pure information-gain maximization — while removing ambiguity and relative risk reduces it to expected-utility maximization. Both classical objectives are limiting cases of one functional. A 2026 workshop result makes the practical consequence sharp: minimizing expected free energy is exactly equivalent to solving a ρ-POMDP whose utility is expected information gain, with the exploration weight fixed at 1 — because the pragmatic and epistemic terms are expressed in the same units, nats. Compare VIME, where information gain is added to the reward with a coefficient someone has to tune.

Resist the temptation to call that one fewer hyperparameter, though, because it isn't — it is a relocated one. The explore/exploit balance reappears in how sharply you specify the preference prior over outcomes: a peaked preference distribution buys you exploitation, a flat one buys exploration, and choosing that sharpness is a design decision of exactly the kind VIME's coefficient is. What active inference genuinely offers is that the trade-off is stated once, in the generative model, in units that make the two terms commensurable — rather than twice, in a reward function and a bonus weight that have no common currency.

The caveats are real and belong here. Millidge et al. argue that expected free energy is not simply "free energy in the future" — they construct what they consider the natural extension of variational free energy and show it actively discourages exploration, so the epistemic term reflects a specific functional choice rather than an inevitability. And Da Costa et al. establish that on partially observed MDPs the standard scheme produces Bellman-optimal actions only at a planning horizon of 1; recovering optimality at longer horizons requires the recursive "sophisticated inference" variant. Elegant unification, still-hard computation.

Where This Lands in Production

The clearest sign this loop is reaching production is what the frontier labs now think is worth measuring. Alongside Gemini Robotics 2 (July 2026), DeepMind introduced ASIMOV-Agentic, a benchmark for "agentic safety orchestration and uncertainty resolution." Read what it scores: whether the embodied-reasoning agent refuses unsafe tool calls from the action model, whether it can predict that a task is impossible before attempting it, and whether it proactively requests human intervention when uncertain. No scores accompanied the announcement, so this is a statement of what the lab intends to be held to rather than a demonstrated capability — but the intent is itself the news. Abstention has graduated from research curiosity to something a frontier robotics stack expects to be graded on.

And the honest counterweight, because W33 should not end on an advertisement: Confidence-Gated Robot Autonomy (May 2026) asks when uncertainty-based gating actually helps, and the answer is "less often than you'd hope." Below a dataset-dependent competence regime, the uncertainty signal ranks errors weakly and unstably; above it, softmax heuristics, MC dropout and ensembles produce similar gating behavior, and the threshold choice matters far more than the uncertainty method. Its evidence base is temporal activity-recognition benchmarks rather than robot execution, so read it as a caution about act-or-defer decisions generally. It is still a fitting place for this arc to end. Four weeks of machinery, and the decisive engineering variable turns out to be where you put the line — a decision no posterior makes for you.

What We Build This Week

Two CPU notebooks make the argument concrete:

NB 00 — Uncertainty-aware MPC. We reuse the GP from W28 as a transition model and plan by propagating the posterior, PILCO-style, then ablate it: same planner, same budget, point estimate instead of distribution. The point-estimate agent confidently plans straight through the region the model has no data for.
NB 01 — Conformal action gating. We calibrate a distribution-free error bound on held-out rollouts, gate execution on it, and compare against gating on ensemble disagreement. On a toy latent model we construct the Biased Dreams pathology deliberately — steering rollouts into a densely-sampled region so disagreement collapses while true error climbs — and measure which gate catches it. We are staging the failure rather than claiming it emerges spontaneously at this scale; the point is to make the two signals separable enough to compare.

Coming Next

W33 closes Arc 2. We started by asking what a model doesn't know, and end with an agent that spends that knowledge — planning through it, bounding it, and abstaining when the bound says stop. Next round opens Arc 3 with the question a single agent cannot answer: what happens when the other thing in your environment is also optimizing? Game theory meets deep learning.

This tutorial is part of the Artifocial research-multimodal series. Previous: W32 Trend Tutorial — Reasoning on Purpose. Next: Game Theory Meets Deep Learning (coming soon).

Build with AI is opening to early users. If you want to put calibrated planning, conformal safety bounds, and the rest of this series to work in your own projects, join the waitlist →