Designing Inclusive Experiences that Go Beyond the Screen

Nearly all mainstream apps and websites rely heavily on the use of text to provide their users with context, onboarding, and notifications. But text-heavy user interfaces (UIs) exclude many user groups from engaging with them, including people who are functionally illiterate, severely dyslexic, and people with visual impairments.

Over 20% of the world is functionally illiterate, and 43% of the United States population has low literacy. Visual impairment affects another 3.4 million people in the U.S. alone, and reading disabilities like dyslexia may affect up to 15% of the population. Users with any of these impairments may be unable to interact with graphical user interfaces (GUIs) that use a lot of text, especially if the text is not accompanied by easily recognizable icons. Icons can provide context and may, when designed with high contrast and simple shapes, be easier than text to distinguish for people with visual impairments.

That’s why I designed an alternative to text-dependent GUIs. Download the Text-Free UI Kit for Adobe XD to create your own truly accessible experience, and read on for more details on why it’s so important to create text-free experiences.

Current technology and opportunities for advancement

Specialized assistive technologies, like screen readers and text-to-speech software, exist to help those who cannot read fluently interact with apps and websites. However, screen readers can be hindered by a multitude of aspects: the lack of alt-text on images, visual captchas with no audio alternative, and text on top of images, which diminishes an app or site’s usability for those who rely on them.

With advances in technology available today, like voice interfaces and augmented reality, text is no longer a required component when creating a usable UI. Voice-first user interfaces allow people who struggle with illiteracy or dyslexia, as well preliterate children, to engage with apps and websites by using voice commands. Mediated reality–a kind of augmented reality that adds helpful information while removing potential visual distractions–can help those with visual impairments interact with a GUI by providing clear instructions through images and animations without requiring the user to read.

Introducing the Text-Free UI Kit for Adobe XD

After learning about the accessibility gap left by mainstream UIs and experimenting with voice prototyping in Adobe XD, I was motivated to try to build a text-free UI kit. The first question I had was: What would that even look like? Every UI kit I had previously encountered relied heavily on text-based UI conventions: in the headers, breadcrumbs, input fields, and descriptive text. The big design problem that concerned me was how to communicate information and functionality to users without relying on text. Even if I could find icons or images to replace the text, how could I design a UI that felt usable and familiar?

To try to answer these questions, I looked at the existing research. A few studies have looked at text-free user interfaces, though they did not include publicly available wireframes or UI elements. I relied on some of the key takeaways from those studies when designing my text-free UI kit:

• Non-abstracted icons and semi-abstract icons are easier to understand than abstract icons, especially without the benefit of text labels
• Provide speech playback wherever possible

Numbers are okay to include in the design, even though words/text are not.

The aim of the UI kit was to provide a blueprint for designers who wanted to design beyond the screen, while also emphasizing that a text-free experience could be usable and engaging.

My Text-Free UI Kit consists of two concepts for text-free experiences, both of which make use of voice cues and AR elements. The first concept, Polaris, is a mobile navigation app that helps users find their way to their destination through the combination of voice commands and AR directions.

The second concept, ARtie (pronounced “Artie”) is a virtual assistant for mobile phones that helps users identify unfamiliar items and locations and solve common problems they may encounter in their day-to-day lives.

There is also a library of distinct icons and components to help designers get started creating their own voice and AR experiences–loading screens, listening screens, and symbols to get started building menus.

With the proliferation of tools like voice and AR, designers can begin to create experiences that are more inclusive and accessible, while pushing the boundaries of traditional UI. Not only will this mean groups of people that were previously excluded can become part of the user base, but it will also provide a more varied and flexible experience for all users. This UI kit leads by example in showing two experiences that go beyond the screen, thereby providing a more inclusive UI for those who cannot interact with text-based UIs.