How a USD workflow brought the “End of Summer” project to life

In 2019, Casimir Perez, Wes McDermott and Vladimir Petkovic, of the Art & Development department within the wider Adobe Substance 3D team, began to consider working on their first collaborative project, a realistic landscape scene. The project was just beginning to move along when the Covid pandemic and subsequent lockdowns hit, and progress slowed right down. One outcome of this was that, when things started to pick up again, more artists joined the project, and it began to expand beyond the scope of its initial concept, a fairly simple 3D illustration of a romanticism landscape.

Casimir: The Marbles RTX demo made by the NVIDIA Studio team was a huge inspiration to us, and one of our goals was to create the same kind of demo internally, within the Adobe ecosystem, and specifically using the Substance 3D tools.

I was leading the project, and Gio and I oversaw artistic direction for the project — we shared the same interest in stop motion animation and we both love the work by Laika Studio. So we wanted to create an homage to all the hard work and craftmanship involved in the making of a stop motion animated feature.

We started to design how this world could look, with the help of John Liberto, the awesome concept artist. We envisaged the warehouse and all of the props surrounding the sets. As the project moved forward, Gio, Carol, and John took charge of character design, and I took the lead on environment design.

The final pitch idea of the project was, ‘The journey of a small family going back home to their windmill after a long day of working in the field.’

With all that in place, we continued the pre-production and I started to list all the props/sets that we needed, and I began making design boards, and so on. Here, it helped a lot that I’d previously worked in animation studios — I had a good idea of the kind of pipeline necessary. It was also a big help that we had Carol on hand working on characters.

Starting production

Production began. The project really became a worldwide production; we had artists working in India, Europe, Canada, Chile, the US, and elsewhere. Because of the different workflows and software used by the team of artists, we needed to find a universal way to store the final assets. Thankfully, Pixar’s Universal Scene Description (USD), a modern and robust file format, was available and we started to heavily leverage this technology for this project.

In addition to the Adobe Substance 3D apps, we used a range of software tools — Maya, 3ds Max, Blender, Cinema4D, and others — storing our files on the cloud for access across the team. The Substance 3D Assets library was key to set dressing the warehouse environment — around two-thirds of the assets in the film come from the 3D Assets library.

To make the scene as realistic as possible, we were very strict about the real scale of assets, thickness of shapes and bevels for the angles. In the finished scene, every asset is as close as possible to real-world size.

Geoffrey Rosin — who's since left our team to work at Valve! — created the warehouse. He did an awesome job using modular pieces to put everything together.

As mentioned, Carol Cornils created the characters, using ZBrush and Substance 3D Painter. I’ll let Carol talk about that in more detail below.

Wes McDermott created the windmill, using the Substance 3D Designer modeling graph. This in particular made iteration very easy, helping us rapidly achieve the precise look I’d been envisioning for the windmill.

The tree in the scene was created by Vladimir Petkovic, using a beta version of our current Substance 3D Modeler tool.

And I did the miniature sets, using a beta version of Substance 3D Modeler and Blender.

The rest of the team created the remaining props you can see in the scene. In total, our team created 400+ individual assets, over about 3 months of work (from modeling to texturing). About 20 people in total worked on the film at various stages — though during the heavy production period, lasting a few weeks, there were just three of us actively working on the film, and we did about 80 percent of the total production.

All assets were textured in Substance 3D Painter and exported as USD files — see below for more information on this.

Character creation

Carol Cornils is a freelance 3D character artist. Carol created the characters for End of Summer.

Carol: I worked on the father and daughter characters at the same time, with a lot of input and feedback from Gio. Notably, the initial shapes were based on circular forms for the dad, and a slim, straight line for the daughter. But that initial idea that the daughter was a bit too tall for her age ultimately made her seem a little older than planned; to counter this, Gio essentially shrank that character from the neck down, making her look cuter and a little rounder.

To maintain contrasting shapes between the daughter and her dad, I then changed his shapes to straighter and heavier lines. For the daughter’s hair, I originally thought about doing something in wool, like you see on the mane of the donkey, but looked kind of messy. The shape of her hairstyle, in the end, is a couple of flowers with petals made of eucalyptus bark.

When it came to the animals, we briefly considered the option of showing internal wires and hinges found in conventional stop motion animation. I initially worked with both dog and donkey with 1-to-1 real life equivalents, then gave them a more ‘cartoony’ look — avoiding falling too much into the trap of making the donkey look like the one in Shrek.

The dog character was the only one that I actually drew in 2D, in Photoshop, before creating him in 3D. My original concept was very close to the final design, though Gio gave me some feedback that the dog’s legs should be thinner.

The level of research that went into the various materials on this project was quite fun. For example, I wanted to better understand how felt is created for the animal's fur, so I learned how to make my own felt. Explained very simply, you take fleece and special needles — and then you stab at the wool, for hours and hours, until the fibers come together in the way you want. It can be a good way to release stress, if necessary.

I created a few different felt samples this way, with various fiber orientations, then took photos of these materials. I used these photos as references to recreate the materials in Substance 3D Sampler. With this range of textures, I was able to play around a little to give life to the characters — with just one felt material, there’s a risk of creating a sort of ‘uncanny valley’ effect with the character, where he looks slightly off.

I also did a few embroidery samples for the daughter’s clothing that Geoffrey Rosin later used to create special brushes in Substance Designer.

Assembling the scene using USD and Omniverse

Casimir: As mentioned above, this project depended heavily on working with USD framework, an open-source technology developed by Pixar Animation Studios: it is very efficient, robust and scalable. It allowed us to have a unique workflow to transfer assets from software to other software.

Omniverse USD Composer (previously named Omniverse Create) was key for us to finalize the project on time. Because we were using a wide range of tools, we needed a hub to centralize all the assets, and carry out the set dressing for the project. Omniverse USD Composer proved to be that hub — we found it to be the best software to learn USD — one with a whole bunch of software features to allow us to assemble the scene very efficiently, such as painting referenced assets on a surface, or dragging and dropping assets that can be snapped to surfaces. And Omniverse USD Composer incorporates NVIDIA’s PhysX physics system, which allowed us to do things like simulate gravity to drop assets on the floor, or on desks, or inside boxes.

I learned this technology the hard way, by bulk editing USD files by hand. Ultimately, it was fascinating to see how easy it was to write and edit snippets of code without knowing other coding languages. Wes, Justin, and I tested out a bunch of workflows before finding what worked for us. It was a challenge for us to find a workflow that worked well for our pipeline — but a fun challenge; the possibilities using USD are endless! We ended up with this type of hierarchy of files.

We used a beta version of Substance 3D Painter to export our assets as USD files. Then Omniverse USD Composer was helpful to create and compose the different sets: the warehouse, the stop-motion set, and so on. We also created assets that referenced a range of other USD assets, such as a pot containing pencils, or a box filled with paint cans.

The final USD stage was composed of more than 6000 assets, and yet weighed only 2.5Gb with textures!

After we’d finished all the set dressing for the scene, it was time to start lighting, and positioning the cameras.

Furry looking

To accurately create that stop-motion ‘look’ I carried out some research to accurately create the look of the felt and fur for the environment elements. I then used Blender to create hair curves that mimicked the style I wanted to have. It was a lot of fun to play with the different features, such as using brushes to trim and comb, or force fields to randomly orient the strands of the grass. I initially tried to stick to real world thickness value of the strands and fibers, but ultimately the various strands are slightly thicker than their ‘real world’ equivalents to optimize the scene as much as possible.

Lighting the scene in Unreal Engine 5

For this project, we wanted to try the capabilities of Unreal Engine 5. Also, because Omniverse USD Composer didn’t support curve rendering at that time (it’s since been updated — it does now), it was a great opportunity to test out and play around with the beta version of UE5. Prior to this project, we had very little knowledge of the software.

As we’d loaded the scene as a USD stage, we could iterate in Omniverse USD Composer and it would update in Unreal Engine 5. That was really helpful. The fur was imported as alembic objects and the characters were imported within the UE5 level.

Placing lights in UE5 with Lumen is a joy — it’s just delightful seeing all those bounces and volumetric effects happening live. We then used the UE5 sequence editor to create all the shots: editing the sequence in real time with the music was pure pleasure.

I was able to render the final images using only my laptop (an HP Zbook G7)! And I used Adobe After Effects to make the final adjustments.

Parting perspectives

This project was such a fun experience for us, to experiment with new tools, technologies and workflows. The biggest challenge was to work collaboratively during the pandemic lockdowns — and with some patience and some helpful online tools we finally did it! Putting End of Summer together has been a long journey for us — but we’re super-happy with how the film came out, and this is driving us to do more projects like this in the future.