‘My Home, My Mind’: A hauntingly beautiful journey with Adobe Aero at the Festival of the Impossible

Every year, the Festival of the Impossible presents an exciting opportunity for digital creators to take audiences on an epic journey via emerging technologies. This year, with so many of us working from home under social distancing guidelines, this exhibition of digital art has taken on a new meaning. With the theme “Identity and the Meaning of Home,” artists were challenged to rethink what home really means to them and to create augmented reality (AR) experiences with Adobe Aero that challenge others to do the same, without leaving their own physical homes.

Adobe creative directors Vlad Petkovic and Justin Patton created their own entry for the Festival of the Impossible. Their creation, “My Home, My Mind,” takes viewers on a journey through a person’s memory, showing significant moments in their life that shaped their identity. Using Adobe Aero’s new sound triggers feature, they were able to create a hauntingly immersive experience sure to impact everyone in a unique way.

Watch the video showcasing the immersive experience, “My Home, My Mind,” created with Adobe Aero.

“We wanted to create a kind of emotional journey through old memories and end it with a statement that there is much more beyond the worlds we create in our own minds,” Vlad says.

Read on to learn more about “My Home, My Mind” and the technology that went into creating it.

Using AR to communicate emotion

“My Home, My Mind” takes viewers through a series of rooms, filled with iconic childhood objects and fantastical surprises around every corner. To strike the balance between “haunting” and “beautiful” that Vlad and Justin were going for, they did sculpting tests in Medium and emulated clay in Zbrush, using these materials to create a look for the room interiors with a “dreary and dusty feeling of the past,” which truly comes alive in AR via Adobe Aero.

As you walk through rooms, you are struck by a feeling of familiarity – a child’s bedroom, the holding (and pulling away) of hands, the growing of trees – all done in a greyscale clay texture. It is melancholic, nostalgic, slightly scary, and definitely impactful.

Behind the scenes with Justin Patton and Vlad Petkovic as they create “My Home, My Mind” with Adobe Aero for the 2020 edition of the Festival of the Impossible.

“The piece serves as a reminder that there will always be good and bad things about our pasts that are absolutely worth exploring, but we should never be constrained by them. There is so much more possibility beyond our lives as we build them in our minds as long as we are open to accept them,” Justin says.

“So much of our own inspiration came from being excited about the new expressive possibilities Aero gave us and feeling like we were part of a new type of medium for creating and sharing art. I hope people come away with that same feeling,” he adds.

The importance of audio in creating immersive experiences

When creating immersive experiences with AR, compelling visuals are key to grabbing the viewer in. But visuals are not the entire story – human beings rely a lot on sound to understand the world around us. That’s why sound triggers were added to Adobe Aero in June. For Vlad and Justin, using sound to trigger emotions was a central part of building the “My Home, My Mind” experience.

“You can communicate so much more by using sound design and music in a piece,” Justin says. “So we began doing lots of experiments with how we could set up sound triggers and emitters to make the world respond to how the user might navigate through it and it made it that more engrossing.”

Together, they were able to use Aero to design and iterate on their experiences, playing with how sound is treated in the AR experience. The end result is a compelling audio experience – atmospheric music plays, a pot shatters, and the opening of a creaky door leads to an exciting moment where trees burst through the floor.

“People are able to see, hear, and even walk through the scenes we have imagined. This way, an observer turns into an active participant in our art installation, whose actions are bringing the entire environment to life,” Vlad says. “As they venture through it, using their mobile devices, they are actually triggering sounds and animations as they are approaching interest points. The biggest advantage is that you can try all these experiences from the comfort of your home.”

Tips for creating effective AR experiences with visuals and audio

AR is a powerful tool for creating experiences and expressing yourself artistically. Vlad’s largest piece of advice to other creators out there is to “dream big.” Since tools such as Aero allow you to break through barriers you may have faced in the past, you should first come up with the core idea of what story you’re trying to tell, independent of any technology constraints you think you might face. Additionally, he offers these tips for creators wanting to break into AR:

• Think about the art direction: What will your experience look like? Is it a 3D scene or a 2D illustration? Is it realistic or abstract? As soon as you settle on the overall style, it becomes much easier to start blocking it out.
• Work in segments: It is often easier to break the entire experience into smaller, digestible parts and then assemble them together in the end.
• Prototype complex interactions before trying to integrate them into the scene: For example, if you want to use a proximity triggered spatial sound, try how it works using very simple objects.
• Know the limits of your platform: We are always trying to push the boundaries of what is possible, but you should know this is a double-edged sword. On the one hand, the biggest reward for us is to stay true to the original idea we have envisioned; on the other hand, working on the edge of technical limits is often tedious.

On that last point about understanding technical barriers, Justin says that AR is particularly well-equipped to overcome any limitations in both creation and distribution.

“It’s amazing to emulate and augment the world as you see it,” he says. “There isn’t as much abstraction of the medium in AR since you’re working directly over reality. Also, the ability to distribute it widely makes it very exciting for communicating ideas through art.”

“I am hoping this will lead to a new age of art form,” Vlad adds.

To watch “My Home, My Mind” and the other entries for the 2020 virtual edition of the Festival of the Impossible, head over to the festival’s website.