Testing for Presence in an Invisible Interface

John is blind. He accesses the web using a screen reader, so that he can listen to the site content he can’t see. When he’s reading a New York Times article where an image provides contextual information for the text, he relies on the screen reader to describe the image—which it can only do if the image has alt text.

WCAG 2.0, the current web accessibility guideline, includes the recommendation that all non-text content have a textual alternative, such as alt text. However, passing the guideline does not ensure that all users have a positive experience. A gap remains between technical accessibility accommodation and a truly universal user experience.

This is a common challenge, which I see often as a UX Analyst. A large part of my time is spent consulting with development teams on accessibility and universal design standards, searching for design and implementation that is usable for all. It’s a difficult process, in spite of the comprehensive, thoroughly validated accessibility standards at our disposal. The standards are just not enough to ensure a universally positive user experience.

In this article, we’ll review a new metric, presence, which may help bridge the gap and make our applications more universally accessible.

The legal vs. functional gap

As a user advocate, it can be difficult to demonstrate to project leadership and development teams that standards are not enough to ensure a positive experience. Section 508, currently the only accessibility legal standard in the US, is far from perfect. At present, Section 508 is undergoing a refresh that will more closely align it with WCAG 2.0. One issue being addressed is the addition of “functional performance criteria” to the Section 508 standard. The idea here is that the current requirements do not address the difference between technical compliance and functional performance for users with specific disabilities.

Complicating matters further, usability testing, which we rely on to illustrate the value of UX to leadership, is not always helpful in pinpointing usability barriers for users with disabilities.

One of the first projects I worked on in my UX career struggled with this very problem: I evaluated an electronic textbook platform that met legal accessibility requirements, but not functional needs. Students who used the platform could access all of their textbooks for the semester in one place and benefit from features such as note taking, highlighting, and discussion forums. Initial user feedback was positive, and overall the electronic textbook platform was well received. But, one reason the university had selected this platform was the belief that it met accessibility requirements.

I quickly began to see some big holes in the ways that accessibility guidelines were met. The most significant problems I noticed were how a blind user would interact with the application. The product had not been developed to communicate with screen readers. The proposed solution was for blind users to copy the textbook content and paste it into a word processor that was compatible with their screen reader. Technically, with this workaround, users could read the textbook content, but they were not able to access all of their textbooks in a central location, and they couldn’t use the note taking feature. Sighted users simply highlighted a passage from the text and were given the option to add a note. Users with screen readers would have to first note the page number for the passage they wanted to reference and then switch from the word processor back to the e-text platform and manually add a note that included the page number for future reference. That’s a lot more work to just add a note!

While the platform was technically accessible for users with screen reader, the user experience was far from equal. I let the stakeholders know that they would be doing some of their students a disservice by implementing the platform, but I had only anecdotal evidence to validate my findings.

Measuring presence for accessibility

Accessibility is quickly gaining acceptance as a universal necessity, and there is plenty of room for user researchers to move that initiative along. The work that has already been done in advocating for accessibility and standardizing accessible design has created a foundation for innovative ways of improving the accessible user experience. I believe we can compare the sighted and non-sighted user’s experience by adding a new usability criteria to the mix: we can measure presence.

In a nutshell, achieving accessible design means presenting the same content to users with and without disabilities and finding that all users experience the same “sense of presence.” Presence can be understood in a few different ways, but in the context of human computer interaction, the user is experiencing presence of self and presence of the content.

• Presence of self relates to how comfortable a user is with a website’s navigational structure – the user’s awareness of location within the structure of pages and components.
• Presence of content relates to a user’s awareness of the information and components that are presented.

A sighted user maintains a presence of self within a website or application through visual cues, such as a highlighted selected menu item, or the position of the browser’s scroll bar. Both of these help the user to identify their location. A user who is blind and using a screen reader maintains a sense of presence by referencing the HTML page title and moving between ARIA landmark regions on the page. Both of these users have the same need for this sense of location within the application or site structure, but the user who is able to reference visual cues is often provided with a more consistent experience.

Similarly, visual designers give presence to content by applying principles of visual hierarchy and weight to different elements, such as header style, italics, bold, and paragraph spacing, to achieve specific results. A user with a screen reader receives this presence of content through code order, contextual description, and other non-visual presentation elements.

Presence research has been used to evaluate other forms of visual media for decades and many of the methods being used already could be adapted to fit the existing user testing and usability research format. As UX practitioners, we can measure a user’s sense of presence to evaluate if a blind user is getting the same (positive) experience as a sighted user by means of questionnaires.

Sense of presence questionnaires

Currently, there is no presence measurement method specifically for user research. Nonetheless, presence measurement is being used in film and television research, virtual environment research, and telepresence research. These contexts are similar to the web in that they each involve a person interacting with mediated content. The questionnaires, surveys, and ethnographic research tools used in presence research could be adapted and incorporated into usability testing for universal design.

One option is to present a questionnaire to the user after he completes a task, and compare questionnaire responses between a sighted and non-sighted participant. Analyzing results from a “Sense of Presence” questionnaire would demonstrate differences in the two types of users’ experiences. When usability barriers are present for the screen reader user but not for the traditional user, the questionnaire shows that even though the product was accessible according to technical guidelines, the actual user had difficulty using it.

Questionnaires can also be used to drive the test facilitator’s prompting during a think-aloud test. In this scenario, the test facilitator uses questions from the sense of presence questionnaire to prompt user feedback. A facilitator might ask both sighted and non-sighted participants to score their awareness of a button or link on a likert scale where 1 is unaware (low sense of presence) and 5 is highly aware (high sense of presence). A sighted user might unflinchingly score the button a 5 because it is highly visible, whereas a non-sighted user might score the button a 1 because their screen reader did not encounter the button. This kind of information could mean the difference between a non-sighted user purchasing an item or leaving the page empty handed.

The International Society for Presence Research has an extensive list of presence research studies and methods on their website. Two of these methods stand out as good starting points for adapting presence research as a usability criteria, the ITC Sense of Presence Inventory and the Para-Social Presence Questionnaire.

ITC Sense of presence inventory

In 2001 Jane Lessiter, Jonathan Freeman, Edmund Keogh, and Jules Davidoff, all from the Department of Psychology, Goldsmiths College University of London published an article presenting their Sense of Presence Inventory. They wanted to provide a “reliable and valid cross-media presence measure” that could be used to establish a baseline between different presence research teams.

The “cross-media” aspect of this questionnaire lends itself to the a user testing context. A prompt like “I felt like I knew what was going to happen next,” though designed for other media forms, could easily fit into usability testing. The questionnaire creators note that differences in a “user’s perceptual, cognitive, and motor abilities” might cause differences in their sense of presence. In fact, the ITC-SOPI questionnaire was developed to find a way to establish a more universal presence measure across different media types.

Para-social presence questionnaire

University of British Columbia’s researchers Nanda Kumar and Izak Benbasat wrote a paper describing the use of a questionnaire focused on computer users’ “sense of understanding, connection, involvement, and interaction” with a computer interface. By treating the interface as one side of a social interaction they focused on the user’s sense of presence within the content interaction. Their questionnaire includes true/false prompts like:

• ABC.com made an attempt to understand my needs
• ABC.com understood my goals
• ABC.com understood what I was trying to do
• ABC.com influenced my decisions

By focusing on the user’s perception of these components of the interaction, user researchers can look for differences in a sighted and non-sighted user’s responses. Everywhere that differences between sighted and blind user experiences exist is an opportunity to create a more equitable and universal experience.

Accessibility beyond legal requirements

Media tests for presence are a great starting point for creating usability tests that target universal design. There is still much room for improvement in the ways we approach evaluating the differences that exist between users with and without disabilities.

As UX practitioners find and test new ways of improving the screen reader user’s experience, we need to look beyond existing guidelines and regulations so that we can design for the user, rather than for compliance. Here are a few ways to get started:

• Check out A Web for Everyone: Designing Accessible User Experiences to learn about the fundamentals of designing for accessibility.
• Web AIM (web accessibility in mind) is a great place to find information on accessibility standards and evaluation tools.
• The International Society for Presence Research is a good place to start for anyone interested in exploring presence research methods and current studies.
• Stay tuned for the upcoming Section 508 refresh. Soon the US accessibility standard for government and federal IT will be aligned with WCAG 2.0.