Movie Gen can create 16-second AI-generated clips from text prompts or edit your existing footage, all with sound.
Meta has unveiled its breakthrough generative AI research for media named Meta Movie Gen, which includes modalities like image, video, and audio. The research enables you to use simple text inputs to generate custom videos and sounds, edit existing videos, and convert your personal image into a unique video.
According to Meta, Movie Gen performs better on these tasks than comparable models in the market when assessed by humans. The new AI tool is based on a text prompt that can create realistic-looking video clips (with synchronized AI-generated audio) up to 16 seconds in length at 16 frames per second.
As Movie Gen is Meta’s third-wave GenAI, the company expects these models enabling various new products that could accelerate creativity. These models have been trained on licensed and publicly available datasets, for which the company has shared its research paper.
Movie Gen has four capabilities:
- Video generation
- Personalized video generation
- Precise video editing
- Audio generation
Users can create videos from text. They just need to give a text prompt and Movie Gen generates high-definition videos at different aspect ratios. By using this GenAI model, users can create personalized videos.
According to Meta, Movie Gen is capable of producing original clips as well as converting pre-existing videos or making unique videos from images. In one example the company has shown a woman in a still headshot with a video of her sitting in a pumpkin patch and enjoying a drink.
Movie Gen can also edit AI videos by using text prompts. The company provided an example wherein a text request could be used to add accessories, like pompons, to a video.
With Movie Gen, generating audio for videos has become easier as users just need to use video and text inputs. The AI model allows users to create and extend sound effects, background music or entire soundtracks.
Stay Tuned to The Future Talk for more AI news and insights!