ViewDreamer Paper

ViewDreamerA Robot World Model beyond Fixed Camera Setups

Ye Mao1,2,* Jinhui Xiong1 Egor Larionov1 Shiyang Lu1 Krystian Mikolajczyk2 Wei Ye1

1Meta AI 2Imperial College London

*Work done at Meta

Meta Imperial College London
2.99Mmulti-view robot video clips for pretraining
14embodiments with different camera rigs
1–4cameras per sample, in any order
500held-out RoboView-Bench test episodes
67.5%average real-robot success of our WAM

Abstract

Video generation models are increasingly important for robot learning, yet most generate each camera view separately or tile all views in a fixed layout. Robot demonstrations, however, are captured by synchronized cameras whose number, placement, and order vary across embodiments. We present ViewDreamer, a single world model that handles variable sets of cameras by disentangling per-view generation from cross-view interaction. Each view follows the generation path of a pretrained single-view model, and views interact only through shared slot tokens. We stabilize multi-view training with a curriculum that gradually increases the number of views. Cross-View Slot Agreement uses visual foundation models to group geometrically and semantically similar regions across views into shared slots. On RoboView-Bench, which covers one to four cameras, ViewDreamer achieves the best generation quality among the models evaluated and improves cross-view consistency over independent per-view generation. These gains carry over to our world-action model (WAM) and action-conditioned world model (AC-WM). Our WAM outperforms the FastWAM-Joint baseline on LIBERO-Plus, RoboTwin 2.0, and RoboDojo, and on a real Franka robot. Both variants improve out-of-domain generalization to various camera settings and scenes.

−23% FVD

Better multi-view video

FVD falls from 32.93 to 25.41 and cross-view consistency rises from 69.3 to 75.7 against Wan2.2-5B fine-tuned on our data.

+19.5 points

Robust to new viewpoints

When the LIBERO-Plus third-person camera orbits the scene by more than 30°, success rises from 24.6% (FastWAM-Joint) to 44.1%.

2.25×

Stronger real-robot policy

On four Franka tasks, our WAM raises the average success rate from 30.0% to 67.5%, and from 20% to 90% on cubes into cups.

Method

Per-view generation, shared cross-view interaction

Every view keeps the generation path of the pretrained single-view video model (Wan2.2-TI2V-5B). A Cross-View Interaction module after each block is the only path between views, so a single model runs on any number of cameras in any order.

ViewDreamer architecture and the inside of a CVI module
Overview of ViewDreamer. (a) Shared DiT blocks process each view, and a CVI module after every block is the only cross-view path. (b) CVI reads all view tokens into shared slots, mixes the slots over space and time, and writes the result back. CSA aligns the last READ with teacher relations between views.

Inside Cross-View Interaction

One CVI module with three views, four slots, and four latent frames. The patch-to-slot colors are illustrative.

1Cross-View Interaction

After every pretrained block, READ gathers every view into shared slots, MIX passes information across slots and time, and WRITE returns it to each view via a zero-initialized projection.

2Cross-View Slot Agreement

A frozen teacher built from DINOv2 and VGGT-Ω links geometrically and semantically similar regions across views. CSA trains the last READ attention so that linked patches share a slot.

3View-Count Curriculum

Training starts from single views and gradually adds more of them. Cross-view interaction thus grows out of the pretrained single-view model instead of disturbing it, which stabilizes training.

Slots that agree across views

READ slot assignments with and without CSA
Last-block READ slot assignments on one RoboView-Bench episode. Each color marks the same slot in both rows.

Without CSA, slots only partially group regions such as the basket (dashed) or the gripper (dotted), and these groups remain fragmented. With CSA, one region seen by different cameras falls into the same slot.

  • Teacher relations combine DINOv2 semantics and VGGT-Ω geometry and are computed offline.
  • The agreement loss only shapes the last READ attention and is dropped at inference.

Scales with the number of cameras

Between three and eight views, DiT time per sample grows from 38.5 s to 104 s with CVI, but from 52.5 s to 263 s with full cross-view attention. With CVI, each additional view adds a nearly constant 13 s.

Cross-view consistency and downstream policy success also keep improving as the pretraining data grow.

DiT time per sample and data scaling
(a) DiT GPU time per sample on one B200 (80 forward passes, no VAE). (b) Gains in three- and four-view CVC and LIBERO-Plus success over the nested 25% data subset.

Multi-view generation

Consistent futures across every camera

Given the first frame of every view and a task instruction, ViewDreamer predicts five seconds of future video for all cameras. Each block shows one method across all cameras, and ViewDreamer is outlined in blue.

Instruction

RoboView-Bench episodes from RoboCOIN, 81 frames at 16 FPS. Four-view episodes are out of distribution: training never sees four views.

RoboView-Bench

Best generation quality among the models evaluated

RoboView-Bench holds out 500 episodes with three and four cameras. Wan2.2-5B (FT) is fine-tuned on the same data as ViewDreamer and generates each view independently, while the other baselines run zero-shot. CVC measures cross-view consistency between the generated views.

MethodSSIM ↑LPIPS ↓FID ↓FVD ↓CVC ↑

Best value per column in bold.

Downstream transfer

A better prior for robot learning

Our world-action model (WAM) adds CVI to FastWAM-Joint and initializes its video network from ViewDreamer. Trained with the same data and recipe, it raises success in simulation and on a real Franka robot, with the largest gains out of domain.

FastWAM FastWAM-Joint Ours
Robust to new camera viewpointsLIBERO-Plus camera perturbations, success by shift

The gain grows with the camera shift. When the third-person camera orbits by more than 30°, success rises from 24.6% to 44.1%.

LIBERO-Plus camera breakdown
Action-conditioned world modelRoboTwin 2.0, three views, EWMScore (higher is better)

Fine-tuned to follow recorded action trajectories, our AC-WM achieves the highest EWMScore in both clean and randomized scenes.

50.09clean; best baseline 49.45
46.99randomized; best baseline 43.08
3.91lead over Wan2.2-5B in randomized scenes, up from 1.76
−3.10drop from clean to randomized scenes; Wan2.2-5B −5.25

Action-conditioned rollouts

Each AC-WM starts from the real first frame of every view and follows the same recorded action trajectory. Our AC-WM, Wan2.2-5B, and Cosmos3-Nano are all fine-tuned on the same clean RoboTwin 2.0 episodes.

Task

Held-out clean RoboTwin 2.0 episodes at 25 FPS, three views each.

Real robot

Rollouts on a real Franka

A Franka FR3 in the DROID setup with two external cameras and one wrist camera. Each policy learns four tasks from 200 teleoperated demonstrations, and we run 10 trials per task.

External camera 1External camera 2Wrist camera

Instruction

Success rate (%)Cup into boxCubes into cupsStack cupsStack cubesAverage
FastWAM4030102025.0
FastWAM-Joint6020103030.0
Ours8090406067.5
The four real-robot tasks
The four real-robot tasks in the three camera views, each shown at one instant of a teleoperated demonstration.

Citation

BibTeX

@misc{mao2026viewdreamer,
  title  = {ViewDreamer: A Robot World Model beyond Fixed Camera Setups},
  author = {Mao, Ye and Xiong, Jinhui and Larionov, Egor and Lu, Shiyang
            and Mikolajczyk, Krystian and Ye, Wei},
  year   = {2026},
  url    = {https://matchlab-imperial.github.io/ViewDreamer/}
}