Gabriel Barcia-Colombo: Singing the body electric

The experience of life during lockdown inspired the New York-based artist to create Choir: an evocative virtual artwork created using augmented reality authoring tool Adobe Aero

A photograph of a woman interacting with Gabriel Barcia-Colombo's Choir AR installation. The AR characters are superimposed on the image reflecting what the user sees on their phone.

by Jim Thacker

posted on 08-07-2020

For many of us, the experience of life during COVID-19 lockdown has changed the way we communicate. Video calls, previously a source of bafflement or irritation for many, became an essential way to keep in touch. Rather than being a substitute for meeting face to face, interacting with an image on a screen became a normal, even a better, way to connect with family and friends. For an artist whose work often explores the unintended consequences of our relationship with technology, it was the perfect stimulus to create new work.

Gabriel Barcia-Colombo’s latest augmented reality piece, Choir, places you at the center of a virtual choir, their stylized bodies echoing the glitchy images of other people we have become accustomed to seeing online in the past months. As you move among them, the choir begins to sing: first simple sounds, then actual words – something made possible by the new spatial audio capabilities of Adobe Aero, the AR authoring tool the artwork was created with.

In Choir, a tablet becomes a window through which to view the virtual choir, a distanced form of intimacy inspired by Barcia-Colombo’s own experience of COVID-19 lockdown.

It’s a curious experience, both eerie and oddly comforting; one that questions the importance of having a physical presence in the world, but also, perhaps, celebrates the ability of digital technology to bring us together when life conspires to keep us apart.

Art at the frontiers of technology

“All of my work is a response to something that’s going on in society, whether that’s questioning the role of technology, or how our identities are changing as a result of it,” says Barcia-Colombo when we catch up – appropriately enough, via a transatlantic Skype call.

That work includes The Hereafter Institute, created for the Los Angeles County Museum of Art: an exploration of death in the digital age that gave visitors a chance to attend their own funeral, with a eulogy created by harvesting traces left behind on social media. An earlier piece, DNA Vending Machine, exhibited at London’s V&A museum, presented DNA samples as “collectible portraits” of their donors, prompted by recent debate over who owns our genetic information.

New York Minute, a 52-channel video installation in NYC’s Fulton Center subway. Gabriel Barcia-Colombo’s work straddles the boundaries of video sculpture, AR and mixed reality.

Although his early work anticipated key aspects of augmented reality – Animalia Chordata uses proximity sensing to enable video projections to respond to the presence of the viewer – Barcia-Colombo began to experiment with the medium in earnest around five years ago, creating Descent, an interactive video sculpture in which viewers can see digital avatars of his friends through an AR lens, sitting or lying on miniature 3D printed beds.

In 2016, Barcia-Colombo founded Bunker.nyc, a pop-up gallery promoting virtual and augmented reality art that opened at Sotheby’s auction house in New York, and the following year, he began collaborating with Adobe on what would become Aero, its AR authoring and publishing app.

3 Transitions, created during Barcia-Colombo’s Adobe Augmented Reality Residency, anticipates some of the themes later explored in Choir, including the use of sound.

Authoring AR artwork on an iPad

Released publicly last year, Aero enables users to author augmented reality experiences on a standard iPhone or iPad, bringing in stock 3D models or assets created in other 2D and 3D software, and assigning readymade interactive behaviors for them.

“The good thing about Aero is that it makes it possible to [get into AR] without a lot of coding experience,” says Barcia-Colombo. “If you have some experience with Photoshop, you can drop assets in and see how they work in the digital world. The experimentation is really great.”

Another advantage of Aero is that the file format it uses, USDZ, can be viewed natively in AR on iOS devices (but not including interactivity at this stage) . “You don’t have to download anything from an app store,” says Barcia-Colombo. “I could text you a project, and you could check it out in your own space.”

Barcia-Colombo first began working with Aero during his time as Adobe’s AR artist in residence, during which he created 12 augmented reality experiments in 12 weeks. Since then, the app has steadily gained functionality, including the option to draw animation paths with your phone or tablet, to light 3D objects using ambient illumination from the viewer’s surroundings and, in the June 2020 update, to import audio files and have them respond to the position and proximity of the viewer: something that Barcia-Colombo would take advantage of for Choir.

Touching at a distance

The idea of creating a choir of virtual avatars arose from Barcia-Colombo’s own experiences during lockdown, particularly of teaching his classes at New York University’s Tisch School of the Arts, where he is a professor in the Interactive Telecommunications Program.

“It was very odd to see these people that I saw in class every week become virtual characters [in a Zoom call],” he says. “I was thinking that their physical bodies had been removed, and that I hadn’t seen or touched anyone for so long. I wondered how I could form intimate connections again, and one way I’ve always felt connected to people is through their voices.”

Barcia-Colombo composed the music and lyrics for the piece himself, roughing out the melody on his iPad in GarageBand. The recordings were coordinated online, again over Zoom, with the four singers working in a “weird exquisite corpse style,” each laying down two separate tracks over the previous recording in Adobe Audition, to create a total of eight vocal parts.

The avatars themselves are stock models, customized specially for the piece. “The bodies are lo-fi, glitchy, distorted. They bend in odd ways,” says Barcia-Colombo. “I was thinking about what it would be like not to have a physical body any more. How would virtual bodies be represented in a physical space?”

The 3D work was carried out in a mixture of software, including Cinema 4D, Maya and Blender, and combines keyframe animation and motion capture, with Adobe Fellow and Mixamo founder Stefano Corazza providing advice on the workflow. Textures for the models were created in Substance Designer, Adobe’s material-authoring software, with Barcia-Colombo assembling all of the assets and setting up proximity interactions inside Aero.

“I created two versions of the choir: one where they’re already singing around you, and one where they’re still and only start to sing when you approach them,” he says. “As an artist, the idea of virtual characters that respond to your physical presence is really interesting.”

Choir inside Aero. Gabriel Barcia-Colombo praises the app for the ease with which users with limited technical experience can create interactive augmented reality experiences.

Aero: An art show in your pocket

For Barcia-Colombo, the appeal of Aero is its accessibility. Although he describes himself as “not a super-crazy 3D graphics person,” Choir was created in just two and a half weeks, and required no custom coding. “There’s still more work to do when it comes to complex interactivity, but it’s a great entryway to augmented reality,” he says.

That accessibility also extends to the viewer: the file containing the AR experience weighs in at just 60MB – small enough to be shared by text – and can be viewed on a standard smartphone. And it can be shared with anyone, anywhere in the world: something that seems particularly significant at a time when many physical museums and galleries are closed.

“The majority of people [in the U.S.] have access to a phone that can run augmented reality now,” points out Barcia-Colombo. “The technology is allowing us to share artwork in a way that we couldn’t before. Even with much of the world quarantined, I can still send you an art show.”

If you have a compatible iOS device, you can download a version of Choir here.

The following artists collaborated with Gabriel Barcia-Colombo on Choir. Vocalists: Maria Valdes, David Castillo, Antonia Marquee, Richard Vanta. CG leads: Pedro Oliveira, Caio Sorrentino. Motion capture and additional animation: James Zachary, Stefano Corazza. Curated by: Joyce Grimm.

At Adobe, we believe that everyone deserves respect and equal treatment, and we also stand with the Black community against hate, intolerance and racism. We will continue to support, elevate, and amplify diverse voices through our community of employees, creatives, customers and partners. We believe Adobe has a responsibility to drive change and ensure that every individual feels a sense of belonging and inclusion. We must stand up and speak out against racial inequality and injustice. Read more about the actions we’re taking to make lasting change inside and outside of our company.

We also know many people are still impacted by the current COVID-19 crisis and our thoughts are with you. The entire Adobe team wants to thank you, our customers, and all creators around the world for the work you do to keep us inspired during this difficult time.

Topics: Design

Products: Photoshop, Creative Cloud