Perception and Regionalism Influence User Satisfaction

Our Research on Voice Recognition

Because Siri was really the first voice-recognition technology to be integrated into a mobile OS, was the first to have a rudimentary personality, and has had the longest timeline for user adoption, we focused our research on Siri.

We set out to figure out why the disconnection between user expectations and Siri’s capabilities exists and what, if anything, we could do to address it. Therefore, we conducted a three-pronged study as our final project in the Kent State University User Experience Design graduate-degree program:

1. Users participated in a two-week diary study to “try to use Siri as much as possible, for as many things as possible.”
2. iOS users took part in an online survey on their usage and perceptions of Siri.
3. We interviewed iOS users in depth about their usage and impressions of Siri.

Four Key Findings About Voice Recognition

After eight weeks of compiling information about participants’ collective Siri usage from notes that they had scribbled in diaries, 67 completions of our online survey, and over 12 hours of interviews, four key findings took shape:

1. Although the users did considerably more with their smart devices than peer-to-peer communication and information retrieval, they have overwhelmingly relegated Siri usage only to communication and simple information retrieval.
2. They limited their use of Siri, in part, because they did not know enough about what they could do using Siri or how to do it. Plus, they did not want to be “that person, you know, the one talking to their phone.”
3. Users want to know more about Siri’s capabilities, but do not know where to find useful information.
4. When people understand both the strengths and limitations of Siri, they are much more satisfied with Siri’s performance.

Four Personas for Typical Siri Users

Based on our research, we created four personas that represented some of the typical Siri users we encountered:

1. Tricia—A Stay-at-Home Suburbanite who primarily wants improved recognition of scheduling conflicts such as adjacent appointments—as well as navigation that’s integrated into her calendar.
2. Jennifer—A Gadget-Geek Power User who most wants application integration across multiple platforms and enhanced anthropomorphic features.
3. Andrew—A Semi-Tech-Savvy Professional who just wants Siri to recognize what he says and successfully complete simple tasks for him while he’s driving.
4. Ted—An Apple Early Adopter who wants everything Apple manufactures and expects it all to work together seamlessly. Ted lives, breathes, and sleeps iOS and Mac OS.

Tutorials to Address User Needs

We then created a schema for a scalable set of tutorials that addressed these four personas’ relative levels of interest in different aspects of Siri’s capabilities. These tutorials belonged to five primary categories:

1. Communication—These tutorials encompassed phone calls, text messages, and Facetime.
2. Social—These were tutorials on Siri integration with social platforms like Facebook, Twitter, Snapchat, and Instagram.
3. Entertainment—These tutorials focused on Siri integration with the Music, Video, YouTube, Photos, and Camera apps.
4. Scheduling—These tutorials covered Notes, Calendar, Notices, and other similar task-oriented apps.
5. Information—This final tutorial set instructed users on how Siri can collect information for users through Web searches and other resources.

We also suggested that there should be basic and advanced levels of tutorials. The scope of our study did not include generating curricula for these different sets of tutorials, but our suggested categories and levels of tutorials would be a good place to start. All in all, this research project was a very illuminating and valuable experience for us. We used three different research methods that dovetailed nicely and enabled us to create potential solutions to users’ problems. That said, the most compelling aspects of studies are often the unexpected information we can glean from the data we harvest.

Tutorials for Tricia—Stay-at-Home Suburbanite

We determined that the following tutorials relating to each of our five categories would meet Tricia’s needs :

• Communication—Tricia was using Siri only to answer and make phone calls, as well as to read and send texts while she was in the car.  A basic tutorial on communication was sufficient for her needs.
• Social—When running errands, Tricia occasionally wanted to post a status update on Facebook to let her friends know what she was up to. Therefore, we concluded that Siri’s basic social tutorial would meet her requirements.
• Entertainment—Since Tricia interacted with Siri mainly while driving, her voice interactions with Siri’s entertainment functions was limited to using simple functions in the Music and Podcast apps. These required only a basic-level tutorial.
• Scheduling—With various scheduling demands imposed by Tricia’s hectic schedule and her coordination of her own and her two kids’ activities, Tricia needed a tutorial set that included both basic and advanced tutorials on scheduling. These tutorials needed to demonstrate how Siri could detect scheduling conflicts and how to integrate the Maps app with her Tasks and Calendar entries.
• Information—Like most kids, Tricia’s kids are impatient and inquisitive. So she needed a tutorial that showed her how to request basic information from Siri while on the go.

Tutorials for Jennifer—Gadget-Geek Power User

Since Jennifer was trying to get as much out of Siri as possible, she needed advanced-level tutorials for all five categories.

Tutorials for Andrew—Semi-Tech-Savvy Professional

Andrew needed the following tutorials on his iPhone:

• Communication—Andrew used Siri in the car to stay connected with clients and coworkers and obtain project information while going from one meeting to the next. Since he needed to remain connected constantly—even in hands-free environments—he required an Advanced Communication Tutorial.
• Scheduling—Since Andrew found himself dealing with complex interactions with clients and coworkers, as well as with project information, he really needed Siri to help him sort out his confusing schedule. He needed an Advanced Scheduling Tutorial.
• Information—Even though Andrew rarely used Siri for anything other than communication, we suggested that he might find a Basic Information Tutorial useful.

Tutorials for Ted—Apple Early Adopter

Ted wanted to maximize the integration of all his Apple devices and to be able to use them as effectively as possible. To achieve this, he felt that using all of Siri’s capabilities to the utmost was an absolute necessity. So, much like Jennifer, Ted needed all five advanced tutorials.

Our UX Research Uncovered Unexpected Regionalism

The most interesting and surprising finding of our research was that there appears to be a rather significant difference in the adoption of voice-recognition technologies across different geographic regions. Alesha lives in California and Scott in Ohio, and the findings from our user interviews regarding Siri usage differed significantly.

First, in Ohio, it was much harder to find Siri users—period. Each of the Ohio interviews showed a combination of lower usage rates overall, as well as more limited usage in terms of the variety of tasks to which users turned to Siri for help. The usage of most Ohio interviewees was for simple dictation in a closed environment—for example, speech-to-text tasks in a car. The Ohio users were relatively satisfied with Siri’s capabilities. Their expectations tended to be somewhat fundamental, and they found that Siri usually executed those basic functions well.

In California, it was much easier to find Siri users, and there were extreme differences in users’ satisfaction with Siri. The California interviewees had experienced using Siri for a wider range of specific tasks—such as opening other apps, making reservations or appointments, or finding directions. Plus, they used Siri in public locations more—at the grocery store, in a restaurant, or while walking to their car—as opposed to just while in their car. In tandem with their greater range of Siri usage and expectations for Siri’s capabilities, their satisfaction levels ranged from outright detestation to enthusiastic adoration of Siri.

Both the Ohio and California interviewees had similar socioeconomic, employment, and educational backgrounds. The only real difference among the interviewees was geography.

The results from our online survey provided minor indications of regional, or geographic, differences, but these were inconclusive because location was an optional field in the survey.

Our study led us to ask an important question: as UX researchers how can we discover differences in participants on a regional, or geographic, basis beyond just collecting demographic data?

It was just a lucky coincidence that we ended up conducting our study in two separate locales and that there were such strong indications of regional differences. Should our personas also have taken regionalism into account? Should Tricia be the Stay-at-Home Suburbanite from the Midwest and Paula be the Stay-at-Home Suburbanite from the Deep South? If we generated personas at that level of specificity, how might that affect the resulting UX-design decisions?

We both intend to continue exploring regional differences in our UX work and to share some of our future findings. For now, we continue to be amazed by both the capabilities and limitations of Siri and are carrying on with our effort to come up with new ideas for tutorials for each of our four personas.