Text-to-video
Text-to-video is a type of generative AI that creates video clips from a written description, or prompt. The model generates frames that match the scene, style and motion described, and many tools also accept an image as a starting point (image-to-video). It is used for marketing, social content, prototyping and storyboards.
Most text-to-video systems extend diffusion models from images to video, generating a sequence of frames that must stay consistent over time. Research such as Video Diffusion Models (Ho et al., 2022) and Make-A-Video (Singer et al., 2022) established early approaches. Tools typically produce short clips, with controls for aspect ratio, camera movement and style, and some also generate audio.
Why it matters: video is expensive to produce, so text-to-video tools can cut the cost of drafts, ads and social posts. Tools differ in clip length, resolution, motion quality, consistency of characters between shots, editing controls, credit-based pricing and commercial-use rights. Compare these using the vendor's documentation and sample outputs that match your use case.
Example: a marketer writes "slow aerial shot of a coastal city at sunrise, warm light" and receives several short clips to choose from for a campaign intro.
Limits: generated video can contain physical errors, distorted hands or text, and objects that change between frames. Clips are usually short and use many credits, and realistic footage raises consent, likeness and disclosure obligations.
Related terms
Learn it step by step
Sources
- Video Diffusion Models. Ho et al., arXiv, April 2022
- Make-A-Video: Text-to-Video Generation without Text-Video Data. Singer et al., arXiv, September 2022
Topic: Choosing AI tools
Last updated