How NIH built the mobile PathogenAR app with the help of Adobe XD
Image source: Office of Cyber Infrastructure and Computational Biology at the National Institute of Allergy and Infectious Diseases, part of the U.S. National Institutes of Health
Simplifying complex scientific concepts is challenging during the best of times. But when the world is thrown into uncertainty because of SARS-CoV-2, the virus that causes COVID-19, finding ways to explain the disease, the potential cure, and the consequences to the public takes on a whole new level of difficulty and urgency.
That’s where biology data visualization comes in – and more specifically, the Bioinformatics and Computational Biosciences Branch of the Office of Cyber Infrastructure and Computational Biology at the National Institute of Allergy and Infectious Diseases (NIAID), part of the U.S. National Institutes of Health (NIH). Comprised of biovisualization specialists, researchers, designers, and developers, the team brings understanding of diseases to life through three-dimensional (3D), virtual reality (VR), and augmented reality (AR).
“We work closely with researchers to build models that visually demonstrate complicated structural concepts to other researchers, and that help communicate scientific concepts to the public and the media,” says Meghan McCarthy, Ph.D., program lead for 3D printing and biovisualization at NIAID.
Image source: Office of Cyber Infrastructure and Computational Biology at the National Institute of Allergy and Infectious Diseases, part of the U.S. National Institutes of Health. Photo by Mark Makela.
But carrying those 3D structures around is not always possible, nor practical, because of their fragility and weight. That’s why the team developed PathogenAR, a mobile application that offers a portable version of high-quality, scientifically accurate 3D models related to infectious diseases.
A finalist in the 2020 Adobe Creativity Government Awards, the app’s interactive nature helps researchers review data and collaborate more easily in-person or remotely — while helping people who are interested in science better understand and retain information.
“If you have a phone and a pocket, you have access to the 3D pathogens. Our goal was to make it as accessible and engaging to as many people as possible.”
Kristen Browne, 3D modelling and medical imaging specialist, NIAID
Visualizing the relationship between viruses and vaccines
PathogenAR was inspired by 3D prints the team was already creating for Dr. Barney Graham, deputy director of the NIAID Vaccine Research Center, to help him explain how vaccines are created.
“Dr. Graham wondered if we could create these structures using AR so that he could just pull them up on his phone, rather than having to carry 3D prints representing aspects of a single structure with him wherever he went,” says team member Philip Cruz, Ph.D., a structural biologist at NIAID. “That was the impetus for this project.”
The team’s first release was the influenza hemagglutinin module in June 2020. They then took their early learnings and applied them to the development of a specific SARS-CoV-2 protein structure called the S protein, which plays a significant role in COVID-19 infections when it sticks to the virus’ host cells.
Image source: Office of Cyber Infrastructure and Computational Biology at the National Institute of Allergy and Infectious Diseases, part of the U.S. National Institutes of Health
“If we can stop this interaction from happening, we can potentially stop infections, which is critical to developing a vaccine for COVID-19,” explains Browne. “It also helps people visualize how a vaccine works. It’s not a cocktail of liquid of chemicals shooting into your arm. It’s a very targeted, designed molecule that has a specific function to prevent you from getting infected.”
The design process began with Browne storyboarding the flow of information in Adobe XD. She used Adobe Creative Cloud solutions such as Adobe Photoshop to composite 3D maps and edit images and Adobe Illustrator to create icons, all of which were easily brought into XD.
Expressive prototypes make it feel like the real thing
PathogenAR features two interactive modes. The first is mid-air mode, which enables users to place a 3D object directly into any space captured by the camera on their mobile device; and merge-cube mode, which allows users to capture and “hold” the model for rotating and zooming.
Image source: Office of Cyber Infrastructure and Computational Biology at the National Institute of Allergy and Infectious Diseases, part of the U.S. National Institutes of Health
Seamless integration among Creative Cloud apps made it easy for Browne to access the files from Photoshop and Illustrator that she needed in XD to build out a prototype. Expressive prototypes built in XD give the team – and the subject matter experts – a feel for what it will be like to use the application.
“As I was finishing up the most recent module, I realized that there was just too much content,” she says. “In your head these things seem like they will work out, but having a working prototype helps you re-evaluate the direction things are taking. At the low-end Adobe XD saves hours by giving us quick and easy ways to send the prototypes for review with some of the interactivity in place. On the high end, it can save days when drawing out the prototype makes it clear that something in the navigation or storytelling just isn’t working.”
XD also helps bring concepts to life for researchers. “In the early stages, it was difficult for subject matter experts to really understand what the app is or what it can do,” says Browne.
“Having a prototype from Adobe XD gets researchers excited about participating. It also can help them brainstorm their own ideas that they might like us to incorporate into the design.”
Kristen Browne, 3D modelling and medical imaging specialist, NIAID
Endless possibilities in a post-pandemic world
Before working on PathogenAR, Browne had never used XD, but she found it intuitive to learn and use. And it’s become the go-to tool for her peers as well.
“I know of many medical illustrators who are doing app design are using Adobe XD now,” she says. “It’s getting a very favorable response from a very critical audience.”
Browne and the team plan to expand the app beyond biological structures. For example, they’re considering a 3D representation that demonstrates how a nasal swab bends to reach deeply into the sinuses for COVID-19 testing. And although their focus remains squarely on COVID-19, they know there’s much more they can do with the app.
“We’re looking forward to presenting at conferences and working with STEM educators to explore other uses,” says Browne. “The possibilities for scientific education and communication are endless.”
Download PathogenAR from the AppStore
Download PathogenAR from GooglePlay
See more examples of creativity in government by viewing the Adobe Government Creativity Awards (AGCA) entries here.