Top

Why Are Contextual Inquiries So Difficult?

Practical Usability

Moving toward a more usable world

A column by Jim Ross
June 4, 2012

Of all the user research techniques, I think contextual inquiry is the most difficult to perform effectively. Despite what you may have learned about doing contextual inquiries in school, from books, or from articles on the Web, they’re not as easy as they seem. When you first try to conduct a contextual inquiry yourself, you’ll soon discover all kinds of unanticipated problems.

Contextual inquiries require a difficult balance between traditional interviewing and ethnographic observation. The name contextual inquiry is foreign to most people outside the field of user experience, and people don’t understand what this approach involves, leading to a lot of misconceptions. In this article, I’ll discuss the most common problems you’ll face when conducting contextual inquiries and how to solve them.

Champion Advertisement
Continue Reading…

What Is a Contextual Inquiry?

The Usability Professionals’ Association’s Usability Body of Knowledge, [1] defines a contextual inquiry as follows: “A semi-structured interview method to obtain information about the context of use, where users are first asked a set of standard questions and then observed and questioned while they work in their own environments.”

The key differentiator between contextual inquiry and other user research methods is that contextual inquiry occurs in context. It’s not simply an interview, and it’s not simply an observation. It involves observing people performing their tasks and having them talk about what they are doing while they are doing it.

Why Are Contextual Inquiries More Difficult?

Another key difference between contextual inquiry and other user research methods is that participants must take a more active role in leading their session. This is unfamiliar territory, and it can be uncomfortable for some people. The dynamic of interviews and focus groups is more familiar to participants, who take a more passive role, sitting back and waiting to answer a facilitator’s questions. In contrast, a contextual inquiry requires participants to take the role of an expert, leading the session by demonstrating and talking about their tasks. For those who are used to taking a more traditional, passive role during interviews, this role-reversal can be a difficult adjustment. Without intending to, participants often slip back into a passive, interviewee role.

Even less familiar user research techniques such as usability testing and ethnography are easier for participants to relate to, because they can take their cues from the researcher—answering questions or performing tasks as requested. The facilitator leads the session, and the participant’s role is fairly clear. In an ethnographic study, being observed can feel awkward at first, but it doesn’t require participants to do anything other than to perform their usual tasks and try to forget about the fact that they are being watched. Unlike a contextual inquiry, they don’t have to explain what they are doing or answer any questions, so it’s impossible for the session to lapse into a traditional interview.

Problems Researchers Encounter in Contextual Inquiries

Because contextual inquiries are so different from other user research techniques, they have their own set of challenges. In this section, I’ll discuss the most common problems and provide solutions for overcoming them.

Clients Don’t Understand What Contextual Inquiry Means

Most people have never heard the term contextual inquiry—and let’s face it, it’s an odd term. In one sense, that’s a good thing. Clients realize they don’t know what it means, so they ask you to define it rather than making incorrect assumptions. But even if you can get your clients to understand contextual inquiry, it’s difficult for them to explain it to others, so they may describe it to participants as an interview or meeting, setting incorrect expectations.

Solutions

  • Clearly explain to your clients what a contextual inquiry is and how it differs from an interview.
  • To avoid confusion, you may want to use the term contextual interview or user research session.
  • Provide your client with text to use in an email message when recruiting or introducing the concept to participants.
  • Once your client has recruited the participants, send another email message directly to participants, introducing yourself and again explaining what will happen during the session.
  • Emphasize that the sessions should be with one participant at a time, that it’s important to observe participants performing tasks, and that the sessions should occur in the environment in which participants normally perform those tasks.

Clients Don’t Understand That Contextual Inquiries Are Flexible Sessions

Clients often don’t realize that contextual inquiries are very flexible, unpredictable sessions that need to flow naturally in whatever direction participants take them. Since they think that it’s an interview, clients often get too caught up in providing specific questions that they expect you to ask participants.

Solutions

  • Ask your clients to define what they want to learn from the research in general, instead of focusing on specific questions to ask.
  • Clarify that a contextual inquiry is flexible and does not follow a specific set of questions or tasks.
  • Instead of showing your clients a discussion guide with specific questions that you’ll ask, show them an agenda of topics that you may discuss with participants and tasks that you may observe.

People Want to Do Sessions in a Conference Room

Sometimes clients or participants don’t understand the importance of observing task performance in context. Clients may schedule sessions in a conference room, thinking that’s more convenient and rationalizing that participants can either use a Web or intranet application from anywhere or connect remotely to their desktop computer. Sometimes participants think that there’s not enough room in their cubicle, or they don’t want to disturb their coworkers, so they decide to move the session to a conference room.

Solutions

  • Explain clearly from the beginning of a project that contextual inquiries must take place where participants normally perform their tasks.
  • If participants won’t do a session in their usual context, politely cancel their session. This sends a clear message that you cannot do a session unless you can observe participants in their natural environment.
  • If your client still insists on doing sessions in a conference room, change the name of your research activity to user interviews and explain the consequences of not seeing the context of the tasks.

People Try to Do Group Sessions

Sometimes clients or participants try to turn a contextual inquiry into a group session. Clients may want to include extra people, thinking they’ll get more feedback. Sometimes you’ll arrive to find a participant has invited another person such as a coworker into their session. From previous experience with business analysts, they may assume that it’s a group requirements gathering session. They may think, “We all use the system. So we should all be there to provide our feedback.”

However, there are times when it does make sense to have more than one person in a session. If two people perform a task together or their tasks closely interact with each other, it might make sense to involve both of them, so you can see exactly how their activities are interdependent.

Solutions

  • Clarify with your client at the beginning of the project—and with participants once your client has recruited them—that contextual inquiries are individual sessions, and you will not be conducting group sessions.
  • If you suddenly find yourself faced with an impromptu group session, politely ask the other people to leave. If appropriate, offer to schedule them for additional sessions with you on their own. If they won’t leave, cancel the session. This sends a clear signal that group sessions are not acceptable when following this method.
  • Be flexible and adapt to situations where it might make sense to have a second participant, especially if the two participants perform interrelated tasks.

Sessions Are Not Long Enough

Sometimes it’s difficult to fit in all of the tasks that you need to see during a single two-hour session—and two hours is usually the limit on the time you can expect participants to spend in one session. When you begin to run out of time, participants may rush through or skip certain parts of the process, trying to fit everything in.

Solutions

  • Carefully plan the tasks that you want to observe. Ask your clients how long those tasks usually take, and add extra time for discussion.
  • If you need more than two hours with a participant, schedule extra sessions to avoid fatiguing the participant and to observe tasks when they normally occur.
  • Divide the tasks you need to observe between participants. It’s better to see one or two tasks in depth during each session than to try to get an overview of many tasks with each participant—and not seeing any tasks in depth.

Participants Keep Slipping into Interview Mode

The hardest part of a contextual inquiry is getting participants to actually perform tasks while talking to you about what they are doing. This requires them to take the lead in the session rather than sitting back and waiting for you to ask them questions, as they would in a traditional interview. The key to a contextual inquiry is that participants act as experts who are educating the researcher about their tasks. Some people are uncomfortable with taking the lead, while others are simply so used to interviews that they can’t help slipping back into passive interview mode.

This problem is especially difficult because most contextual inquiry sessions do start out with a traditional interview to gain background information about the participants and their tasks.?Unfortunately, this brief question-and-answer interlude can set a pattern, making it difficult to get participants to start performing their tasks. During the session, as you ask questions about what participants are doing, it’s common for them to stop doing and start answering, then wait for further questions instead of going back to doing.

Solutions

  • In your introductory email message to participants and at the beginning of a session, explain that this is not an interview and that it’s very important to observe complete tasks as participants normally perform them.
  • After your initial interview questions at the beginning of the session, make a dramatic transition to the observation of tasks—such as, “Now we’re going to do something completely different….” This makes it clear that the session is not going to be a traditional interview.
  • Explain and keep reminding participants that they are the experts in the tasks they perform, and you need them to show you and talk about each of the steps that they take in a process.
  • Often begin your sentences with, “Show me….”

Participants Don’t Show You Complete Tasks

Even when you do get participants to perform their tasks, it’s difficult to get them to give you a complete demonstration. Sometimes participants don’t understand that you want to see every step of a process. So, they’ll give you a short, summary version of a task and talk about what they do instead of actually doing each step.

In other cases, participants don’t have any real tasks to perform during a session. For example, you may want to observe how they create a monthly report, but your session does not occur at the time of the month when they normally create that report. Or you may want to see how they would normally complete a transaction, but they don’t have any to show you and can’t submit a fake transaction.

When participants don’t have any real tasks to show you, they often try to demonstrate how they would normally do a task, but this can become a false exercise, in which participants demonstrate some steps, omit others, and simply talk through the rest. The session becomes more of a discussion and easily lapses back into interview mode.

Solutions

  • Once your client schedules participants, contact the participants and clearly state how important it is for them to be able to demonstrate real tasks from beginning to end during the session.
  • In your first contact with participants, ask them to define the tasks they perform and to save some of those tasks to perform during the session.
  • If possible, schedule the sessions around the times that participants normally perform the tasks you want to see.
  • When participants can’t show you a task at the time when they would normally perform it, ask them to do their best to simulate the task—for example, creating a fake report—and simply talk you through the parts of the task they can’t show you.

Participants Talk Above Your Level of Understanding

Especially when you’re conducting user research on an unfamiliar domain, it’s important to get participants to talk to you at a level that is appropriate to your understanding. Unfortunately, it’s difficult for participants to step outside of their expert knowledge and avoid speaking in jargon.?

Solution

  • Before the sessions, learn as much as you can about the subject matter.
  • Remind participants about your lack of knowledge of the subject matter.
  • Frame the session properly, communicating that the participant is the expert, and you are there to learn from him or her. In a work situation, you can ask participants to think of you as a new employee who has no knowledge of the company or the work they do. This expert/apprentice model is a good frame of reference for how they should conduct a session.
  • If necessary, ask dumb questions to remind participants how much you don’t know.

Contextual Inquiries Turn into Complaint Sessions

In work situations, participants sometimes expect a contextual inquiry to be a discussion about the problems they’re encountering with an application or product. They are used to talking with business analysts who document their complaints and requests for new features. You know you’re in trouble when you arrive at the session and they pull out a long list of problems to discuss with you. Often they have polled their colleagues to gather this feedback.

These lists are valuable, in that they usually document the most frustrating usability problems, but the danger is that these complaints may take over the entire session. Some participants are very insistent about showing you these problems and constantly turn away from demonstrating tasks to return to their list of complaints. While it’s useful to know the problems they face, you may run out of time to observe their tasks. You end up learning a lot about the problems with the application, without knowing much about the participants or the tasks they are trying to accomplish.

Solution

  • Clarify that the purpose of a session is not to communicate a list of all of their problems with an application. You do want to hear about those things as they come up in the course of observing a task, but the main purpose of a contextual inquiry is to understand their tasks.
  • If participants want to discuss a list of problems, ask them to save that discussion for the end of the session.
  • If necessary, ask participants to give you the list when you leave, then schedule a follow-up session or call to go over the problems.

Not Enough Time for Analysis

Most people consider themselves lucky enough to get user research into a project at all. There’s rarely enough time to go through the user research process at a leisurely pace. It usually takes a lot of time to identify and schedule participants, and the sessions themselves take up a lot of time. By the time you get to analyzing your findings, you’re often rushed.

Analysis of contextual inquiry data can be extremely time consuming. This approach to user research generates a huge amount of data. And since it’s difficult to take complete notes during sessions, you usually need to review recordings and type up more detailed notes. For every hour of research, it takes at least two hours to listen to recordings and type your notes. Then, you have to organize your notes into themes, consider their implications, and make your recommendations. Finally, you have to create a deliverable to communicate the findings to the project team and your clients.

All of this takes a lot of time, and rushing analysis diminishes the quality and value of the research. It’s a waste of time to do extensive research, then spend insufficient time analyzing the results.

Solutions

  • Estimate a realistic amount of time for analyzing the results—at least two hours for each hour you’ve spent on the research sessions.
  • Allocate time for researchers and designers to collaboratively synthesize the findings.
  • When time is tight, eliminate the creation of a formal research deliverable, instead spending more time on analysis and applying the research during the early stages of design.
  • Bring designers along with you to observe contextual inquiry sessions directly. Seeing the research first-hand eliminates the need to transfer knowledge later. Afterward, both of you can quickly begin to analyze the findings and apply them to the design.

Be Flexible in Your Methods

Although contextual inquiries can be difficult to conduct, they are an extremely valuable way of gathering information about users and their tasks. Don’t worry too much when you encounter problems. Accept that it’s rare for everything to go perfectly, and try to think of these problems as learning experiences that can help you to refine your application of this method of user research in future engagements.

Regardless of which user research method you use, your goal is to understand users and their tasks. We often get too caught up in choosing the perfect method and performing it correctly. In reality, the best approach often involves a combination of methods. Almost every user research session consists partly of an interview. Sometimes it’s most appropriate to do a contextual inquiry, while at other times, simply observing people perform their tasks uninterrupted gives you the best information. Be flexible in combining these methods as appropriate to get the information you need. 

Reference

[1] Usability Professionals’ Association. “Contextual Inquiry.” Usability Body of Knowledge, 2010. Retrieved on March 29, 2012.

Principal UX Researcher at AnswerLab

Philadelphia, Pennsylvania, USA

Jim RossJim has spent most of the 21st Century researching and designing intuitive and satisfying user experiences. As a UX consultant, he has worked on Web sites, mobile apps, intranets, Web applications, software, and business applications for financial, pharmaceutical, medical, entertainment, retail, technology, and government clients. He has a Masters of Science degree in Human-Computer Interaction from DePaul University.  Read More

Other Columns by Jim Ross

Other Articles on User Research

New on UXmatters