Closing the gap between computers and creativity

One of my favorite TV shows growing up was The Jetsons. It’s still amazing to see how cartoon writers in the 1960’s foreshadowed so many of the technologies we use today, such as video calls and robot cleaners. Although I do wish we had flying cars...

Robotics and computer programming have been passions of mine since I was a child, eventually leading me to earn a bachelor’s degree in electrical engineering. During my studies, I took a course on digital image processing that connected those two passions and helped me understand how robots could perceive and process the world through pixels. I immediately jumped on the Computer Vision wagon and 11 years later, I am just as inspired by the technology’s potential. We still have so much to discover.

What I do

I’m a scientist and I create content and edit videos. Computer Vision comes down to the way human beings think and solve problems. It might sound counterintuitive, but that’s what attracted me to the world of artificial intelligence. As a Research Scientist at Adobe, I help our product teams develop advanced computer algorithms that solve real-life problems for our customers and help them do their jobs better and more efficiently each day.

What’s even cooler is that I’m able to solve problems that don’t even exist yet. My team operates on the edge of human knowledge, pushing boundaries and imagining business’ needs five or 10 years down the line. We consider not only how people think today, but also imagine how new technologies will change the way they work in the future.

That, believe it or not, makes the research work I do incredibly creative. At its scientific core, creativity is about connecting neurons and synthesizing huge volumes of information into something new. That process happens on two levels in my role: In my mind, when laying the groundwork for new research projects, and in the AI models I help to build and integrate into Adobe’s solutions. The work is abstract, though rooted in the practical needs of the people who use our solutions in the real world.

Creativity is intrinsic to where I focus at Adobe, which is video content creation and editing. Beyond my technical expertise, I need to understand the ins and outs of making films and videos. I need to identify with videographers and editors, know what makes them tick, and empathize with what drives them crazy. After all, it takes a well-rounded view of your end-user to deliver a well-rounded app they will use over and over to get their jobs done.

It’s not always easy to develop technologies for creatives and content creators. The work of an engineer or scientific researcher has traditionally been worlds apart from that of a designer or marketer. We need to understand how our findings can be used in the real world to deliver the best possible solutions.

Thankfully, the gap is closing fast. One of my research projects involved developing a system that automatically cuts video into plausible sequences to save time and effort for content teams. We started the project thinking this was a far-off goal, but after working closely with our customers we realized we were closer than we thought, and we were able to come up with a viable solution to a very real creative challenge.

Moments like that, where I had the opportunity to design and implement solutions to hard problems, are what get me out of bed in the morning. My work sits at the heart of what’s next in the world of video creation and the potential impact it could have is incredibly exciting.

What’s a research brainstorm?

Another creative part of my role is information gathering (think: surveys, interviews, etc.), which is essential when using AI responsibly to solve real-life problems. I’m lucky to work at a company full of talented people from every background imaginable, and that makes our approach to problem-solving especially well-rounded and diverse.

I’ve always been encouraged, especially at Adobe, to communicate with people outside my own field — be it artists and designers, product engineers, or other experts — to approach research with a broad perspective and mindset.

An Adobe research brainstorm is a lot like a traditional creative brainstorm. We get people from many different disciplines together in a room to discuss a new idea — and make it better. We poke, prod, add, and augment our research approach until it’s as focused as possible and will measurably help to improve the solutions for our customers.

There are few things more inspiring than brainstorming with people with a different education or background than my own. We all bring something unique to the conversation, and when it’s working, we’re also self-aware and we understand that it takes more than one point of view to see the world.

Decoding the art of visual storytelling

One project that I’m particularly energized about is understanding patterns in the video editing process, which will eventually help make it easier for editors to cut and sequence movies. My team and I are essentially decoding the anatomy of video storytelling, using the power of machine learning and AI to mine how visual stories come together.

For instance, we might look at how the right color palette, camera angle or soundtrack will amplify the suspense of a scene in a horror film. Some of that stems from cinematographic theory, but experience of what works is just as important. By programming machines to understand those best practices we can help more creators put out impactful video that resonates at an emotional level.

Crucially, our goal is to democratize creative tools, not sacrifice artistic integrity. Another reason I love my role at Adobe is that we have no interest in replacing people with algorithms or watering down creativity. Our goal is to augment the content creation process for people so they’re able to enhance the content consumption experience for their audience.

I love defining and taking on tough challenges, especially when my contribution will have an impact on content creators for years to come. It makes my role as an AI researcher at Adobe the perfect fit for me. I get the opportunity to set up ambitious, exploratory moonshot projects that test my technical skills while making creativity more accessible and user-friendly each day. It’s a truly inspiring position to be in.