World in World: Explore the World with World Models
Mirrored from Hugging Face Daily Papers for archival readability. Support the source by reading on the original site.
World in World: Explore the World with World Models
Abstract
A training-free interface enables flexible camera and time control in frozen autoregressive video world models by routing heterogeneous visual evidence through native self-attention with correspondence-guided queries and per-channel attention guidance.
Autoregressive video world models enable interactive, long-horizon exploration, but flexible control remains challenging. Exploring a source video from new viewpoints requires the generated rollout to remain synchronised with the recorded event, place observed content in the requested view, plausibly complete newly exposed regions, and recover previously generated appearance on revisits. Existing methods typically address these requirements through task-specific modules or additional training. We present World in World, a training-free inference-time interface that converts heterogeneous control evidence into camera- and time-labelled clean visual states, which are read through the native self attention of a frozen causal video model. The evidence comprises source-video observations, target-view scene projections, geometry renderings that guide completion of newly exposed subject regions, and retrieved generated states beyond the rolling cache. Each evidence source carries token-level support and its own availability schedule. A correspondence router combines persistent point identities with geometry to establish token correspondences, guiding supported queries towards matching source-video tokens. Evidence-wise attention CFG (EWA) then independently regulates each auxiliary channel's additional contribution using attention responses from the same denoising forward pass. The shared interface supports camera-controlled rerendering, long-horizon revisiting, and human-motion transfer with the same frozen backbone. We evaluate World in World on camera-controlled video rerendering under diverse viewpoint changes, assessing perceptual quality, temporal consistency, and camera-following accuracy.
Get this paper in your agent:
hf papers read 2609.11548 curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper
No model linking this paper
Datasets citing this paper
No dataset linking this paper
Spaces citing this paper
No Space linking this paper
Collections including this paper
No Collection including this paper
More from Hugging Face Daily Papers
-
CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents
Sep 17
-
Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches
Sep 17
-
In-Context Robot Learning with VLM Agents
Sep 17
-
Assessing nnU-Net Generalization across Brain Tumor Populations in BraTS-GoAT 2026
Sep 17
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.