Text or image-to-video Driven by the success of text-to-image diffusion models, generative video models are able to generate short clips of video from a text prompt or an initial image. These models extend a pretrained diffusion model to generate videos by adding some type of temporal and/or spatial convolution layer to the architecture. A mixed dataset of images and videos are used to train the model which learns to output a series of video frames based on the text or image conditioning. This guide will show you how to generate videos, how to configure video model parameters, and how to control video generation.  Popular models Discover other cool and trending video generation models on the Hub here! Stable Video Diffusions (SVD), I2VGen-XL, AnimateDiff, and ModelScopeT2V are popular models used for video diffusion. Each model is distinct. For example, AnimateDiff inserts a motion modeling module into a frozen text-to-image model to generate personalized animated images, whereas SVD is entirely pretrained from scratch with a three-stage training process to generate short high-quality videos.  Stable Video Diffusion SVD is based on the Stable Diffusion 2.1 model and it is trained on images, then low-resolution videos, and finally a smaller dataset of high-resolution videos. This model generates a short 2-4 second video from an initial image. You can learn more details about model, like micro-conditioning, in the Stable Video Diffusion guide. Begin by loading the StableVideoDiffusionPipeline and passing an initial image to generate a video from.   Copied import torch