Building intuitive AR experiences that matter with the Smithsonian, the Hydrous, and Adobe Aero

The growth of augmented reality is enormous— millions of people use augmented reality (AR) functionality every day to access facial filters, visualizations, and games. It’s important to remember, though, that AR is still an emerging technology and we’ve really only started to scratch the surface of what’s possible on our mobile phones. With four billion camera-enabled mobile devices around the world, the opportunities are staggering.

At Adobe, we are excited to explore how this new medium can connect the physical world with the digital. As a technology that can empower creatives to make beautiful, rich, and meaningful art in a more intuitive way, we have started to observe incredible organic experiences that are transforming the world around us and the minds of those who experience them.

Today, we’re honored to announce an exciting partnership rooted in that transformation, as Adobe, the Smithsonian, and the Hydrous partner together to bring you a one-of-a-kind augmented reality project to experience remote learning.

Dive into the ocean with the Smithsonian, the Hydrous and Adobe Aero

This collaboration was initiated to create an interactive, immersive experience that raises awareness about the threats facing ocean ecosystems like coral reefs. It also underscores the complexity and importance of marine habitats to the health of our planet and ourselves — a beautiful story of how we are all connected. This initiative is part of a program, led by the Hydrous, called The Decade of Ocean Empathy. It has been endorsed under the UN Decade of Ocean Science for Sustainable Development.

To make this project happen, we brought together a true dream team:

• The Smithsonian opened up their incredible 3D collection (open access/CC0) and provided some original samples of scanned corals.
• The Hydrous Chief Scientist Dr. Erika Woolsey worked with her team, including Science Advisor Celine de Jong, to build an engaging, science-based story.
• Our talented team at Adobe brought it all to life in Adobe Aero, our creative AR toolkit.
• Ocean advocate and citizen scientist Danni Washington came on board to narrate the story.

The project showcases a series of interactive, 3-dimensional animations to help viewers factually understand and visualize marine life with photorealistic renders and digital duplications of the real element. It’s not just about visualizing a 3D model in AR, but about experiencing a narrative with deep interactivity, which is key to the learning process. This experience enables anyone with a supported mobile device to be immersed in a rare and special environment, learning about a crucial topic in the process.

Immerse yourself among the corals from the comfort of your home

Discover the experience for yourself with these QR Codes by downloading the Aero app. We recommend running the experience for iOS on 8S and above, or on Android, Private Beta, US only, a list of Android can be found here on HelpX. (FYI, the experience may take a few seconds to load as it is a more sophisticated AR project.)

To give context on how this experience was created, the 3D scene contains original, scanned data of the corals from the Smithsonian collection. These models have been optimized and incorporated with other 3D elements, such as the coral reef and animated marine life to create a realistic and immersive environment. The scene is brought to life using textures, which have been created in Substance Painter, along with narration, music, and other sounds.

Finally, the scene features a variety of interactive moments, made using the behavior system in Aero. This allows for elements like infographics, animations, buttons, and audio to play when a viewer interacts with the scene, enhancing the educational component even further.

All data used in the experience has been scientifically fact-checked by marine biologists from the Hydrous team.

Redefining educational experiences with interactive AR

As AR is evolving, we’re exploring its power in communicating our history, natural world, and science. A museum is a beautiful sanctuary with physical content, but what if we could expand it with digital, true to life content, so it’s no longer reserved to the few? At the same time, only a small percentage of the museum’s collection is shown at a given time, which shows the potential of removing physical barriers.

Research shows that immersive and interactive learning experiences can impact feelings of presence, agency, and empathy in the learner. For this reason, public institutions like the Smithsonian have started to adopt AR technology to put viewers at the core of the experience.

In education, it’s critical to ensure a message can be fully integrated. By creating an empathetic mindset and immersing someone into an experience, we empower them to receive that message in the best possible way.

AR can also be used to provide close access to unique environments people would not easily be able to experience, like facing a coral reef at the bottom of the ocean. In addition, the technology can show an element being transformed (like corals coming back to life), which would not be possible by displaying static fossils in a static display.

AR is the natural next step in democratizing education and sharing immersive experiences around the globe with the potential to create a positive impact.

New Aero features and improvements based on direct product feedback

The Smithsonian Aero partnership is rich in many ways — including the feature development insights gained during production on the project. The richness of this experience helped the Aero product team test and validate many of the new features soon to be released that will help AR creators to seamlessly build complex sequences of actions in a more intuitive way because of new abilities to directly manipulate editing controls and now instantaneous visual feedback while putting a scene together.

• Drag-and-drop behaviors: Re-order actions through simple drag-and-drop functionality to quickly adjust the order of actions without having to delete and re-enter settings.
• Parallel action groups: Combine multiple actions into a single group that will finish playing back as one block before playing the next actions in a sequence.
• View-all behaviors: Toggle between scene and object level view of behavior sequences, so you can see both the big picture of a complex scene, and a focused view of all actions related to a given asset.
• Replace asset: Quickly swap out placeholder content with updated assets and retain placement and behavior settings from the original asset.
• Smoother iOS viewing flow: View AR experiences without logging into Aero and go straight to the AR content.

These refinements to the publishing and viewing flows remove friction and enable you to iterate more efficiently, ensuring those interactive moments feel just right.

Aero at Adobe MAX

Check out all the AR-related content at the free, virtual Adobe MAX 2021 (October 26-28):

• Augmented Reality: How It’s Transforming the Space Around Us (with Carrie Gotch)
• Rethinking Educational Experiences with Interactive AR (with Vladimir Petkovic and special guest Dr. Erika Woolsey, founder of the Hydrous)
• Creating an Interactive Augmented Reality Art Exhibit (with James Zachary)
• AR Design Principles: How to Create Immersive Experiences (with Heather Dunaway Smith)
• The 3D Asset Pipeline for Augmented Reality (with Don Allen Stevenson III)

Also tune in for inspiring new AR projects and live sessions with artists throughout MAX and beyond on Adobe Aero’s Instagram channel. For more frequent updates follow Aero on Twitter, get inspired on Behance, and be sure to tag your own AR creations with #AdobeAero.

With thanks to the team at the Smithsonian, the Hydrous, as well as the Adobe team (Kimberly Potvin for partnerships, and Vladimir Petkovic for AR experience creative direction) and the team at Bonfire Labs for the content production.