Stable Video 4D (SV4D) is a generative model based on <a target="_blank" rel="ugc" href="https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt">Stable Video Diffusion (SVD)</a> and <a target="_blank" rel="ugc" href="https://huggingface.co/stabilityai/sv3d">Stable Video 3D (SV3D)</a>, which takes in a single-view video of an object and generates multiple novel-view videos (4D image matrix) of that object.<ul><li>Developed by: <a target="_blank" rel="ugc" href="https://stability.ai/">Stability AI</a></li><li>Model type: Generative video-to-video model</li><li>Model details: This model is trained to generate 40 frames (5 video frames x 8 camera views) at 576x576 resolution, given 5 reference frames of the same size. To generate a 5x8 image matrix from a single view video, first run SV3D on the first input frame to generate an orbital video following a specified camera path, then use the orbital video as SV4D's reference views, and input video as reference frames, as conditioning for 4D sampling. To generate longer novel-view videos, we use the first generated frames as anchors, and then densely sample (interpolate) the remaining frames. Please check our [tech report] and [video summary] for details.</li></ul>Model Sources<ul><li>Repository: <a target="_blank" rel="ugc" href="https://github.com/Stability-AI/generative-models">https://github.com/Stability-AI/generative-models</a></li><li>Tech report: <a target="_blank" rel="ugc" href="https://sv4d.github.io/static/sv4d_technical_report.pdf">https://sv4d.github.io/static/sv4d_technical_report.pdf</a></li><li>Video summary: <a target="_blank" rel="ugc" href="https://www.youtube.com/watch?v=RBP8vdAWTgk">https://www.youtube.com/watch?v=RBP8vdAWTgk</a></li><li>Project page: <a target="_blank" rel="ugc" href="https://sv4d.github.io">https://sv4d.github.io</a></li><li>arXiv page: <a target="_blank" rel="ugc" href="https://arxiv.org/abs/2407.17470">https://arxiv.org/abs/2407.17470</a></li></ul>Community License: Free for research, non-commercial, and commercial use by organizations and individuals generating annual revenue of US $1,000,000 (or local currency equivalent) or more, regardless of the source of that revenue. If your annual revenue exceeds US $1M, any commercial use of this model or derivative works thereof requires obtaining an Enterprise License directly from Stability AI. You may submit a request for an Enterprise License at <a target="_blank" rel="ugc" href="https://stability.ai/enterprise">https://stability.ai/enterprise</a>. Please refer to Stability AI’s Community License, available at <a target="_blank" rel="ugc" href="https://stability.ai/license">https://stability.ai/license</a>, for more information.

Files with SHA256: bdfe5bb33dfc771fc102891883befcf061873f4a96fa602037a964beca83cb44