State of AI in Animation

AI challenges creative conventions and puts cutting-edge animation capabilities within reach.

As passionate as animators can be about their craft, there are some jobs that are simply tedious or a means to an end. Animating hair, cloth, walking, and splashing water can all fall into this time-consuming category.

No wonder, then, that Pixar developer Jiayi Chong, whose filmography includes classics like “Toy Story 3,” “Up,” and “WALL-E,” set out to build a tool called Midas Creature that automates these kinds of animation tasks. This automation is made possible by artificial intelligence (AI).

Adobe is doing something similar to tackle syncing a character’s lip movements to a voice recording. “It’s a skilled job,” says Adobe fellow and Academy Award-winner David Simons. “And it takes a lot of time. Because of this, if you’re new to animation production, you might limit what and how much you’ll do.”

These are some of the latest innovations aimed at streamlining and expanding a medium with massive potential — online videos now account for 50% of all mobile traffic, and animated content is a big piece of that digital pie. More importantly, animation is effective and efficient — animated explainer videos have been shown to increase conversion rates by more than 20%.

Despite these benefits, animation has traditionally been time- and resource-intensive, making it prohibitively costly for many. AI is changing that — not only pushing the content bounds but also automating time-consuming tasks so creatives can spend more time creating.

“There’s definitely a big opportunity to make workloads easier for creators,” says Wil Li, principal scientist at Adobe and research lead on Character Animator. “That’s especially true for animators and artists working on animation — many tasks are tedious and aren’t really what artists want to be spending hours and hours of their time doing.”

Do more with less

Animation, traditionally, has gone hand in hand with minutiae that don’t necessarily fuel an animator’s creative prowess. However, these steps are usually necessary — and they’re also complex enough to make automation a challenge. For many animators, the central benefit of AI comes from expediting the creative process by simply taking over tedious tasks. With that added “free time,” designers can spend more time working toward their overarching creative visions.

“When you speed processes up to be so fast, completely new ways of operating are enabled,” David says. “That sped-up workflow enables new artistic capabilities. Really that’s how things have always been. Every time technology has automated a task, humans have then started working on the next thing.”

Push the bounds of creativity

As AI technology continues to refine processes and proliferates through the industry, animators, filmmakers, and designers are diving in, producing high-quality animation with less headcount in less time. This, in turn, drives new creative concepts that cut through the clutter in the existing digital video — and animation — landscape.

A recent example: Google’s Deep Dream is an experimental project that helps scientists and engineers view images through the eyes of a deep neural network. A new form of psychedelic, abstract, and sometimes unsettling art quickly emerged from this technology.

“If it was trained on images of cats and faces, it would create something like an LSD-induced dream, churning out new combinations of those things,” says David. “It produced these new visual things we’d never seen before. Some of it was disturbing, but there’s potential to generate new, unorthodox, creative ideas from it.”

That said, in a sea of highly predictable creative content, the unpredictability AI brings to creative workflows could provide an exciting spark of originality that producers need to stand out from the crowd online — and technology like Deep Dream could fuel that artistic vision in animation.

Simplify the complex

Going a step further, researchers at Allen Institute for Artificial Intelligence and the University of Illinois Urbana-Champaign fed over 25,000 three-second clips from the original “Flintstones” cartoon into an AI system. From there, the AI system was able to create new clips based on text descriptions of a given scene — “Betty and Wilma have a discussion in the kitchen,” for example.

Using intelligence from the existing clips, the AI system assembled the characters, props, and locations specified and animated them accordingly. While some results were clunky, most delivered new content seamlessly and almost instantly.

Clunkiness aside, this research hints at a future in which AI generates hours of animated content based on scripts of past content. This could, immediately, democratize high-quality animation, making it a simple creative option for any marketer, producer, creative director, or aspiring animator — an option that could come with a significantly lower price tag than traditional animation development.

Creative content, though, is just the beginning. This same AI technology potentially can be applied to gaming, motion analysis, and even robotics going forward. “Things are changing pretty rapidly,” David says, “and I think the speed will continue to increase as more people are trained on the technology and think about the best ways to use it.”

That said, most experts still see a gap between the quality AI delivers and the standard of quality expected by animation professionals and studios.

With any kind of automated tools for a creative endeavor like animation, there’s always a bit of tension about what the machine should be doing and what the artist should do. It takes some time for those techniques to get to the point where the quality is high enough for artists to feel comfortable using them.

Helping artists achieve a comfort level with AI includes giving them control over automatically generated results. That’s the direction and the fundamental challenge that we need to meet as we look toward the future — providing effective controls and refining the quality of the results in this domain.

Visit the Adobe Blog for more on how artificial intelligence and machine learning are driving the creative process.