A new era of video editing
The ever-increasing demand for fresh, short-form video content means editors, filmmakers and content creators are being asked to do more and in less time. Today not only do editors cut picture, but they’re also tasked with color correction, titling, visual effects, animation, audio mixing and more. At Adobe, we’re leveraging the power of AI to help editors expand their creative toolset so they can work in these other disciplines, delivering high-quality results on the timelines their clients require.
Common editorial tasks — like navigating gaps in footage, removing unwanted objects from a scene, smoothing jump cut transitions, and searching for the perfect b-roll takes time. When performed well, addressing these tasks can make the difference between a compelling and emotional narrative, rather than one that distracts from the story you’re trying to tell.
Sometimes, sharing creative intent with your team and the stakeholders that green light and fund the work can also be a challenge, requiring many rounds of communication. At Adobe, we provide tools such as Frame.io to streamline teamwork by enabling a uniquely integrated review and approval process. And now, we are delivering AI tools to help facilitate an even better creative process that helps take the tedium out of post-production. Our AI tools give editors more time to explore new creative ideas, the part of the job they love, while also setting them up for successful collaboration with the larger team. Not only does Adobe facilitate streamlined work process, but now, with Generative AI, we’ve made it even easier and faster for editors and motion designers to create their best work in record time. Watch below to see how we’re taking video editing to new heights using Firefly.
With Firefly Text-to-Video, you can use text prompts, a wide variety of camera controls, and reference images to generate B-Roll that seamlessly fills gaps in your timeline.
All generations shown below were created with Adobe’s Firefly Video Model and were generated in under 2 minutes.
Prompt: Cinematic closeup and detailed portrait of a reindeer in a snowy forest at sunset. The lighting is cinematic and gorgeous and soft and sun-kissed, with golden backlight and dreamy bokeh and lens flares. The color grade is cinematic and magical.
Prompt: Slow-motion fiery volcanic landscape, with lava spewing out of craters. the camera flies through the lava and lava splatters onto the lens. The lighting is cinematic and moody. The color grade is cinematic, dramatic, and high-contrast.
Prompt: Miniature adorable monsters made out of wool and felt, dancing with each other, 3d render, octane, soft lighting, dreamy bokeh, cinematic.
The Firefly Video Model excels at generating videos of the natural world. When production misses a key establishing shot needed to set the scene, generate an insert with camera motion, like landscapes, plants or animals.
Prompt: Footage of a camera on a drone flying over a desert with wind blowing over the dunes creating waves in the sand below.
Need more complementary shots? Fill the gap in your timeline with a generated clip based on a reference frame.
Prompt: Detailed extremely macro closeup view of a white dandelion viewed through a large red magnifying glass
With the Firefly Video Model you can leverage rich camera controls, like angle, motion and zoom to create the perfect perspective on your generated video.
Prompt: Drone shot going between the trees of a snowy forest at sunset golden hour. The lighting is cinematic and gorgeous and soft and sun-kissed, with golden backlight and dreamy bokeh and lens flares. The color grade is cinematic and magical.
The more detailed your prompts are, the better the model can leverage that depth to generate inspirational imagery and b-roll.
Prompt: Cinematic closeup and detailed portrait of a man in the middle of a rainy street. the lighting is moody and dramatic. The color grade is blues and teals. the man is extremely realistic with detailed skin texture and stubble on his face. movement is subtle and soft. the camera doesn’t move. heavy film grain and textures. Beads of water slowly rolling down his face.
The Firefly Video Model supports a broad variety of use cases including creating atmospheric elements like fire, smoke, dust particles and water against a black or green background that can then be layered over existing content using blend modes or keying inside Adobe’s tools like Premiere Pro and After Effects.
Ideate 2D and 3D animation, including Claymation, that can be shared with creative collaborators to communicate intent.
Prompt: Stop motion 2d animation made of felt of an egg cooking in a frying pan in a sunlit beautiful kitchen on the stovetop.
Prompt: Hand-drawn simple line art, a young kid looking up into space with a wondrous expression on his face.
Prompt: An adorable kawaii cheeseball on the moon smiling, 3d render, octane, soft lighting, dreamy bokeh, light sparkles floating in the background.
Brainstorm ideas for custom text effects to share with clients for feedback.
Prompt: Macro detailed shot of water splashing and freezing to spell the word "ICE”.
With Image-to-Video, you can bring an existing still shot or illustration to life by transforming them into stunning live action clips.
Prompt: Zoom out of the galaxy to reveal that it is contained inside the pupil of a human eye, detailed macroscape.
And coming later this year to Premiere Pro (beta), Generative Extend allows you to extend clips to cover gaps in footage, smooth out transitions, or hold on shots longer for perfectly timed edits.
Watch how the editor uses Generative Extend powered by the Adobe Firefly Video Model in Premiere Pro to hold on a shot longer to match the crescendo of the audio.
And here you can see the final output with the generated frames creating the perfect edit.
We’re excited by all the recent advancements on the Adobe Firefly Video Model and look forward to continuing to partner with the community to build generative AI into the Adobe tools and workflows you rely on.