0

Imagine360: Immersive 360 Video Generation from Perspective Anchor

$360^\circ$ videos offer a hyper-immersive experience that allows the viewers to explore a dynamic scene from full 360 degrees.

Preview
Year
2024
Venue
arXiv 2024
Authors
7
Hosting
Abstract onlyARXIV-DEFAULT

Cite

Notes

Only stored in your browser.

Attribution

Abstract & full text
arxiv.org/abs/2412.03552ARXIV-DEFAULT
TL;DR
Semantic Scholar
Attribution policy →

Abstract

360^\circ videos offer a hyper-immersive experience that allows the viewers to explore a dynamic scene from full 360 degrees. To achieve more user-friendly and personalized content creation in 360^\circ video format, we seek to lift standard perspective videos into 360^\circ equirectangular videos. To this end, we introduce Imagine360, the first perspective-to-360^\circ video generation framework that creates high-quality 360^\circ videos with rich and diverse motion patterns from video anchors. Imagine360 learns fine-grained spherical visual and motion patterns from limited 360^\circ video data with several key designs. 1) Firstly we adopt the dual-branch design, including a perspective and a panorama video denoising branch to provide local and global constraints for 360^\circ video generation, with motion module and spatial LoRA layers fine-tuned on extended web 360^\circ videos. 2) Additionally, an antipodal mask is devised to capture long-range motion dependencies, enhancing the reversed camera motion between antipodal pixels across hemispheres. 3) To handle diverse perspective video inputs, we propose elevation-aware designs that adapt to varying video masking due to changing elevations across frames. Extensive experiments show Imagine360 achieves superior graphics quality and motion coherence among state-of-the-art 360^\circ video generation methods. We believe Imagine360 holds promise for advancing personalized, immersive 360^\circ video creation.

Authors

7