Behind the Scenes with Adobe Dimension Engineers: How We Built the 3D Publish Feature
3D design is here to stay and is quickly transitioning from a ‘nice-to-have’ to a cutting-edge skill that can set designers (and companies) apart from the crowd. Last year, Adobe jumped into the 3D arena with the release of Adobe Dimension CC, our 3D compositing tool that makes it easy to transform 2D designs into full, real-world visualizations. The power of Dimension is that, rather than directly targeting 3D content creators, Dimension is aimed squarely at graphic designers. These artists are now able to take their 2D designs, created in Photoshop, Illustrator, and InDesign, and easily place them onto 3D models in photorealistic environments. The primary goal of Dimension is to democratize 3D, making it accessible to anyone and, by doing so, streamline workflows that can see a massive benefit from 3D technologies, from product design to synthetic photography.
Sometimes, however, a designer might want to share something more than a 2D render of their creations. What if they want to share their entire 3D composition for someone to explore and review? Or maybe they want to publish it to the web for mass consumption. In these cases, wouldn’t it be great if having a copy of Adobe Dimension CC wasn’t required to view their creation?
Publishing from Dimension to the Web Using glTF
In Dimension 2.0, we’ve introduced a new feature that lets users export their 3D scene to be viewed in a web browser by anyone with a link. This is achieved using glTF.
glTF is a file format designed to enable the efficient transmission of 3D assets between runtimes. It is ideal for representing a real-time version of Dimension’s scenes for several reasons. The glTF 2.0 spec includes a physically-based (PBR) material system that maps quite closely to the one used by Adobe Dimension and glTF is designed to be fast to load. Additionally, support for technologies like Google’s Draco geometry compression allow us to shrink scenes that are often very large to sizes that are more download-friendly.
Exporting Dimension projects to glTF wasn’t without its issues, however. Our users ideally want the real-time versions of their scenes to look as close as possible to what they see in Dimension and several key features that would make this possible were missing from the glTF spec. As part of the glTF Working Group, we’ve been able to solve for these shortcomings by helping to define new extensions to the specification to make sure Dimension users can export their projects with the highest level of quality.
Lighting
The first missing feature was that the core glTF specification doesn’t include any environment information such as scene lighting. The reasoning for this is that glTF assets are most often loaded into an existing scene running inside a 3D engine and so the lighting information is inherited from there. This has the benefit of not bloating file size with information that would just be thrown away. However, this means that someone viewing the scene is always going to be seeing something slightly different than what the creator of the asset saw. With Dimension’s userbase, this is a critical issue. A designer who is building a scene in Dimension probably isn’t interested in exporting their creation into a game or virtual world; they are sharing their work of art and they want the viewer to see exactly what they intended.
There are two types of lighting in Dimension that are required for a faithful representation of the scene. The first is our ‘sunlight’ which not only models the light coming directly from the Sun but also includes a ‘cloudiness’ factor for modeling sunlight reflected around the sky via cloud-cover. We helped in designing and specifying the KHR_lights_punctual extension that includes definitions for directional lights, spot lights, and point lights. Sunlight can be best modeled using a directional light so this extension solves the first piece of Dimension’s lighting requirements. The ‘cloudiness’ factor isn’t included in this extension as it’s specific to the use case of the Sun rather than directional lights in general. Until there’s another way to describe this effect in glTF, we are including this information about cloudiness in a custom tag within the glTF that is only used by our glTF viewer.
The second type of lighting used by Dimension is image-based lighting (IBL). This type of lighting essentially uses an image (often HDR) to store the scene’s lighting from every angle (projected from a sphere onto the object). This image is then used for rendering the light being diffused and reflected from the surface of an object. It’s quite a bit more complicated than it sounds, of course. When light reflects off a surface, its path can be perturbed a little or a lot by microscopic imperfections and this results in rough or smooth-looking materials. Computing this efficiently in real-time isn’t possible so the usual solution is to pre-filter the IBL image for various levels of material roughness, storing a series of images that can be used when rendering the final scene. We worked with the team behind Microsoft’s Babylon.js to define the EXT_lights_image_based extension and are now exporting this data from Dimension. Though the resulting lighting doesn’t look exactly like a high-quality Dimension render, future improvements to our lighting export will help to further close the gap in fidelity.
Left — A 2D image rendered by Dimension. Right — Fully interactive 3D glTF, viewed in a web browser.
Transparency
Although glTF 2.0’s PBR material system aligns quite well with Dimension’s use of the Adobe Standard Material, one area that it’s clearly lacking in is transparency. Only a simple alpha coverage value is supported (exposed via the alpha channel of the ‘baseColorTexture’ image). While this can be used to make a material transparent, it only realistically does so in situations where light is neither refracted nor reflected. A classic example of this is a loose fabric such as gauze. However, far more common transparent materials such as plastics or glass involve much more complex interactions with light. To render convincing glass, for example, light is both refracted and reflected as it impacts and passes through a surface. In addition, some of the light can be absorbed or scattered, effectively changing the colour that reaches the viewer’s eye. For users of Dimension, rendering common materials such as glass, plastic and various liquids is extremely important so being able to recreate these substances in glTF is a must.
For this purpose, Adobe has defined the ADOBE_materials_thin_transparency extension to handle the simplest cases of optical transparency. This extension deals exclusively with thin materials (i.e. no variations in thickness or volumetric effects are modeled) and excludes dynamic scattering effects for simplicity. We will continue to work with the glTF community to incorporate this functionality into a future specification so that it can be widely adopted.
Cameras
Dimension has a feature called Bookmarks that can be used to save particular views of the scene. For each bookmark, designers can configure camera properties such as the field of view (FOV) as well as a focus point and depth-of-field strength. Since glTF cameras only include FOV, we defined another extension to include more advanced camera information. ADOBE_cameras_advanced defines focal length, focal distance and aperture which we use for depth of field effects for each camera.
The Dimension Web Viewer
When a designer publishes their scene from Dimension, it becomes available to view in the Creative Cloud web interface. The designer can then share a link with anyone to give them access to view the project in their web browser using the Adobe Dimension Web Viewer. This viewer is built with Babylonjs and, although it can load and display any glTF (or GLB), it especially excels at previewing Dimension projects. This is because a Dimension project can include features that don’t really belong in a generic scene format like glTF and so we’ve ended up including them as custom data in the published file, only to be consumed by the Dimension viewer. These features include things like static backgrounds as well as an invisible ground plane that can be used to blend object shadows and reflections with the background image.
Aside from recreating what you see in Dimension, the viewer’s features are fairly minimal at the moment. All of the project’s camera bookmarks are available in the viewer to make it easy for the designer to setup and show off specific shots and a commenting panel is available to leave feedback about the scene. We’re always interested in hearing from people about what features they’d like to see so please drop us a line anytime at our Feedback Portal.
Here are some sample scenes.
https://dimension.adobe.com/viewer/index.html?gv=https://dncrshare-us-east-1.cloud.adobe.io/v1/shared_asset.glb?shared_asset_base64_encode=aHR0cHM6Ly9jYy1hcGktY3AuYWRvYmUuaW8vYXBpL3YyL2RpbWVuc2lvbi9hc3NldHMvY2M5NTdiYzAtNzIwYS00ZGM3LWE0ZjktYjc0MDgxZmYyMzI0L29yaWdpbmFsL3ZlcnNpb24vMA==
https://dimension.adobe.com/viewer/index.html?gv=https://dncrshare-us-east-1.cloud.adobe.io/v1/shared_asset.glb?shared_asset_base64_encode=aHR0cHM6Ly9jYy1hcGktY3AuYWRvYmUuaW8vYXBpL3YyL2RpbWVuc2lvbi9hc3NldHMvY2M5NTdiYzAtNzIwYS00ZGM3LWE0ZjktYjc0MDgxZmYyMzI0L29yaWdpbmFsL3ZlcnNpb24vMA==
https://dimension.adobe.com/viewer/index.html?gv=https://dncrshare-us-east-1.cloud.adobe.io/v1/shared_asset.glb?shared_asset_base64_encode=aHR0cHM6Ly9jYy1hcGktY3AuYWRvYmUuaW8vYXBpL3YyL2RpbWVuc2lvbi9hc3NldHMvMjI5NGY5MTEtNjQxMC00M2I3LWEzZTUtOTBjZGJjODUyM2RjL29yaWdpbmFsL3ZlcnNpb24vMQ==
https://dimension.adobe.com/viewer/index.html?gv=https://dncrshare-us-east-1.cloud.adobe.io/v1/shared_asset.glb?shared_asset_base64_encode=aHR0cHM6Ly9jYy1hcGktY3AuYWRvYmUuaW8vYXBpL3YyL2RpbWVuc2lvbi9hc3NldHMvMjI5NGY5MTEtNjQxMC00M2I3LWEzZTUtOTBjZGJjODUyM2RjL29yaWdpbmFsL3ZlcnNpb24vMQ==
https://adobe.ly/2PIyWYa
https://adobe.ly/2PIyWYa
Adobe is committed to continuing to improve Dimension’s export pipeline as well as working with the glTF community to advance the specification and support our user’s needs. Look for further improvements and additions to come to the web viewer in the coming months.
To try out new Dimension features in advance, sign up for our prerelease builds at www.adobeprerelease.com.