Bringing generative AI to video editing workflows in Adobe Premiere Pro

Video is the most compelling storytelling medium. From short-form social posts to feature length films, video gives creatives, brands, and anyone with a phone in their pocket the power to connect and engage. Adobe's suite of video tools – Adobe Premiere Pro, After Effects, and Audition - are the industry-standard for creating professional video, and for over a decade, we’ve delivered AI-powered innovation in editing, sound, color and effects.

We believe generative AI is the next accelerant to human creativity, and Adobe’s strategy is to give creative professionals more power and efficiency by integrating generative AI across Creative Cloud. The Premiere Pro team has been deeply engaged with the professional video community over the past year to understand how generative AI can solve real problems in their work. Editors want to use generative AI to reduce workflow tedium, streamline tasks, and increase creative control, with three important requirements:

• AI must be integrated into their workflows, and the tools they use every day like Premiere Pro.
• The real value is in editing existing content faster with Generative AI – adding objects, removing distractions, or extending clips seamlessly on the timeline. All with your content and with your precise control.
• The power of choice – and to be able to use different AI models that are great at different things.

Today, we’re excited to share our vision for bringing generative AI video workflows inside of Adobe Premiere Pro with brand new tools designed to make shots a little longer, add new objects or remove unwanted objects, and generate missing b-roll. By creating interfaces and workflows that unlock the power of generative AI, professional editors will be able to do things they’ve never been able to before, without leaving Premiere Pro.

Adobe Firefly Video Model and third-party generative AI models

Of course, this vision requires advanced video AI models – and we believe the rapid pace of research and innovation in this space will yield professional level results over the coming year. At Adobe, we are developing a new Firefly Video Model that will join the already released Firefly Image, Vector, Design, and Text Effects Models.

Premiere Pro has long had a rich partner and plugin ecosystem, making it the flexible, extensible tool that fits any workflow. To expand this ecosystem, we are also sharing our early explorations around incorporating third-party generative AI tools and models with Premiere Pro such as OpenAI’s Sora, RunwayML and Pika.

Generative Extend

Need a few more frames? No problem. The Generative Extend tool can be used to make shots a little bit longer by generating entirely new frames at the beginning or end of a clip. Use the extra media to fine tune edits, hold on a shot for an extra beat, or add a few frames to cover a transition. Just click and drag for the perfect edit.

Object Addition and Object Removal

New AI-based smart masking and tracking tools make it easier and faster than ever to select moving objects over time. Combined with the Object Addition and Object Removal tools, editors can now replace moving objects in a shot, remove unwanted items like boom mics or stray coffee cups, or quickly add set dressings such as a painting or a photorealistic flower vase on a desk.

Generative B-Roll

Spend less time hunting for b-roll shots in stock footage and use simple text prompts to generate video clips that are just right for your story. Now you can create a shot that illustrates a difficult-to-shoot or fantastical concept, generate b-roll that’s hard to source, or craft placeholders to assist with planning and pacing.

An open approach to delivering AI models

As we continue to innovate in the rapidly developing world of Generative AI, we’re committed to open platforms and partnerships to deliver the capabilities our customers want where they want them. Through early explorations with companies like OpenAI and Pika, customers will be able to choose the right model for their use cases. And we know all models may be trained be differently, so we are committed to transparency through Content Credentials, which are like a nutrition label for content creation, to ensure consumers will always know what models were used in the creation of the media they are viewing. Content Credentials will be supported inside Premiere Pro, to help create a chain of trust from creation to editing to publication.

AI has the power to supercharge productivity and lower the barrier to work in new mediums. We believe creativity is a uniquely human trait, so we’re excited to continue the dialog we’ve had with the creative community as we enter the era of generative AI in Premiere Pro.