Pushing the Limits of Digital and Physical Realities: Project Glasswing and Beyond

At Adobe, our DNA has always been to explore the limits of how our software can unlock the creative potential with new innovation. We are now entering an exciting new era of interactive breakthrough experiences where the physical and digital worlds are blending and bringing immersive content in 3D, AR and beyond to the real world.

Shown at SIGGRAPH 2019, the Adobe Research team is pushing the boundaries again with an early-stage mixed reality exploration called Project Glasswing. An experimental display prototype, Project Glasswing explores the possibility of bringing Adobe’s design tools to the physical world, creating the illusion of a Photoshop, After Effects or XD layer appearing in space in front of 3D real objects, without the need for phones or special glasses. This touch-enabled experiment enables us to demonstrate new experiences that blend digital content and interaction with real and physical objects, unlocking new possibilities for the 3D ecosystem around the future of creative storytelling.

To get a sense of how this exploration works, check out the video below:

https://www.youtube.com/watch?v=0_Pkbz4eAV8

For example, in-store advertising could allow shoes to be displayed inside a window with accompanying text, graphics, and video to create a compelling experience for customers. A vending machine could use a masked video of sloshing water to augment the view of real-physical bottles of water behind it. Or imagine a museum exhibit of a rare artifact displayed in a glass case. The glass could allow a clear view of the encased item, with overlaying graphics offering information about its history.

This project is just another example that builds on Adobe’s 35+year legacy of inventing core technologies that explore the limits of innovation, ranging from the intro of Adobe Dimension, which allows designers to compose and render 3D product mockups; to AEM (Adobe Experience Manager) Screens, a digital signage solution that enables marketers to create and publish interactive in-store displays; to Project Aero, a powerful augmented reality (AR) authoring tool that makes it easier for designers to blend the digital and physical worlds and easily craft high-quality AR experiences. And just last month, we unveiled the latest innovations around our upcoming interactive drawing and painting app, Adobe Fresco, embarking on an entirely new category of groundbreaking live oil and watercolor brushes invented by the scientists of Adobe Research.

We are exploring how to create content for this exciting class of display that has only emerged in the research community in the last few years. Today’s mainstream displays are typically limited to 2D, but the vision here is to democratize the entire ecosystem of 3D objects. Adding touch input and dynamic imagery to it gives the SIGGRAPH attendees an opportunity to try this unusual form of interactive mixed reality experience for themselves.

With Project Glasswing, digital media assets created in Adobe tools such as Adobe After Effects, Adobe Premiere Pro and Adobe Dimension can now be shown on the display for a new generation of storytelling. By controlling the opacity and emissive color of each pixel independently, the display acts like a conventional monitor or could have transparent windows to see physical objects.

We’ll have much more to share on this journey of exploring the limits of AR and growing the creative canvas of digital-physical breakthrough experiences on the road to MAX. Stay tuned.