A Sneak Peek at the Future: Living and Marketing in an Immersive World

How brick-and-mortar institutions are embracing AR and VR to move consumers beyond the screen.

A café in Japan, the elevator at One World Trade Center, the Smithsonian. As you step into these places, your reality might feel a little unreal. That’s because they’re embracing the newest immersive experiences, ranging from 360-degree video to augmented reality (AR) and virtual reality (VR) — the first wave of a big disruption. Immersive tech is poised to wrench us free from our screens, and stitch tech right into real life. The implications for each of us, especially for the relationships between brands and their customers, are huge.

Moving out of this world.

Gavin Miller, vice president and fellow with Adobe Research, has the perfect analogy for what’s about to change. Right now, we’re in a “Star Trek world,” interacting with devices that connect us to information and each other. But, according to Gavin, we’re on the verge of a “Harry Potter future,” in which, “everyday objects will spring to life, interactive books will write themselves, and portraits will come alive with vivid personalities. Instead of having to search the internet or sift through documents, intelligent assistants will materialize out of thin air to help us with daily tasks.” We’re already seeing the first glimpses of this.

Step inside the elevator at One World Trade Center and engage in an immersive experience that animates the creation of Manhattan’s skyline as you rise to the top. ©2015 The New York Times News Service.

Consider the elevator at One World Trade Center. The walls are lined, top to bottom, with monitors that appear to give you a window onto the city. But as you ascend, you’re transported through time and space. In less than a minute, you’ve witnessed 500 years of change in the city. The experience is knitted into reality, yet completely unreal.

Museums are another spot where interactions have the potential to revolutionize our experiences — right now, a flat website with images of exhibits doesn’t replace the excitement of being inside a gallery, choosing your own perspective. However, The Smithsonian American Art Museum, an early adopter of VR tech for museums, is letting people visit virtually. Their demo of room-scale VR uses laser scanning and photogrammetry to create 3D models from 2D photographs. The result is a reality that feels familiar — you walk through a room and step closer to the art you want to see in more detail. But there are also those “Harry Potter-esque” elements — descriptive text appears when it’s relevant, and there are portals you can step through to get inside a painting.

The Smithsonian is letting people visit exhibits virtually with room-scale VR. Image credited to Intel/VALIS.

Other new immersive experiences focus on delight, while giving us a tiny peek at how the tech might revolutionize marketing. Take, for example, a series of Starbucks gift cards in Japan. They were designed in collaboration with artist Asami Kiyokawa. When you use your smartphone and an AR app, butterflies on the card appear to come to life and fly around you, giving you a chance to snap photos of yourself and your coffee in an augmented, more beautiful, reality.

Marketing beyond reality.

Instagram-ready product moments, like the butterfly gift card, are just the tip of the iceberg when it comes to marketing. According to Mark Asher, Adobe’s director of corporate strategy, “Immersive space opens up a whole new set of advertising possibilities. Can you imagine putting a virtual ad into a live cityscape that someone is walking through, in real time? Maybe you’d look up and see an airplane skywriting ads in virtual space. I can look into the sky and see things that grab my attention, and are relevant to the context of my activity — they’re just stitched into the narrative of my day. It doesn’t feel artificial anymore.”

With this ability to add to reality, Mark envisions how immersive tools could transform day-to-day experiences, making them not just novel, but better. Imagine, for example, going on vacation to New York. You hold up a smartphone as you walk through the street and see everything around you. But you also see which restaurants have tables available, and what their Yelp reviews are; which Broadway shows have tickets, and what the critics have to say about each one; along with directions to your destination.

What makes this such a game-changer for marketers is that everything blends together — it’s not a matter of stopping to enter information into your phone and searching for what you need. Instead, you’re moving naturally and organically through your world, and the information is flowing to you as you need, or want it.

Looking even further into the future, Mark predicts that immersive experiences will make familiar media, like music videos, interactive brand experiences. “Maybe I’m watching a music video in 360 degrees, and I’m able to actually look at the artist’s clothes and touch them, maybe even try them on in virtual space. And then, I could purchase them directly from the video as I’m watching and looking around the environment. If I also want more information on the car they’re driving, or the hotel where the video happens, I can get it. All of that is potentially available in immersive experiences.”

Eventually, Mark predicts users will make even more fundamental choices about the shape of their own experience. “Maybe you’re exploring a region that never existed except in virtual space,” he says. “Eventually we can customize that for each individual. So, you can imagine that if you really like to ride horses, then I, as a marketer, could present a message in the context of an equestrian venue or experience. Someone else may prefer a driving experience, in which case the experience could dynamically shift to a racing environment.”

Once marketers can create these kinds of super- personalized experiences, there will be opportunities to understand relationships with their customers far more deeply. Instead of measuring click-through rates and how long someone stays on a webpage, you can begin understanding how people are engaging intellectually and physically. “You might measure how long people look at a specific ad, or how long they listen to something, or whether they move toward a marketing message. You’ll know if they look up to see your virtual skywriting, or if they turn up the volume. Those are new signs of engagement,” explains Mark.

The possibilities are still being invented, so we’ve only just started to grasp what could be next. “Immersive media is one of the biggest disruptions of the 21st century,” says Mark. “Think about how we moved from desktops to personal computers to smartphones. Immersive media is next.”