User research consists of two core activities: observing and interviewing. Since we’re most interested in people’s behavior, observing is the most important of these activities because it provides the most accurate information about people, their tasks, and their needs.
While interviewing is also very important, the information people provide during interviews isn’t always accurate or reliable. Often, research participants don’t know why they do things, what they really need, what they might do in the future, or how a design could be improved. To really understand what people do, you can’t just ask them, you have to observe them.
But exactly what is observation, and what does it entail? Though we all know what the word observation means and everyone knows how to look and listen, there is more to it than just pointing your eyes in a particular direction, listening, and taking notes. By doing a little research, I found many books and articles about interviewing, but surprisingly few about how to observe research participants. So, in this column, I’ll first explore what observation is and the different types of observation methods, then focus on one particularly useful, yet underused UX research method: naturalistic observation.
Champion Advertisement
Continue Reading…
What Is Observation?
To really understand the concept of observation and better appreciate what it entails, let’s look at some definitions.
“To watch carefully, especially with attention to details or behavior, for the purpose of arriving at a judgment.”
What Can We Learn from These Definitions?
Focusing on the key words and phrases in these definitions, we can see that observation involves the following:
“action or process”—Instead of passively absorbing sights and sounds, observation is a conscious, active process.
“closely,” “carefully and attentively”—Observing something closely requires the careful application of effort and attention.
“notice or perceive something”—Instead of considering all data equally, it is necessary to pick out important elements selectively.
“significant details”—Observation involves actively focusing on the significant details and filtering out the rest of the data.
“purpose of arriving at a judgment”—Observation has a goal: answering specific questions.
So observation is not passively looking and listening. It requires careful, conscious, purposeful effort. We actively direct our attention to certain things, notice particular elements, process the information, and determine the significance of our learnings in answering specific questions.
Types of Observation in User Research
Observing human behavior is an important element of most user-research methods.
Usability Testing
Usability testing involves both observing and listening to participants as they attempt to complete tasks with a user interface. Participants may think aloud, and you can ask questions to better understand what they’re thinking and doing, but the primary value is in observing their actions.
Contextual Inquiry
Contextual inquiry means observing people in their natural environment, as they demonstrate their typical tasks. Research participants lead their own session, explaining what they are doing, but the primary value is in observing the details of the ways they normally perform their tasks.
Naturalistic Observation
In naturalistic observation, the researcher attempts to observe one or more people unobtrusively, without interacting with them. The goal is to observe participants’ natural behavior, without interrupting them or affecting their behavior.
Shadowing
In shadowing, the researcher follows participants around as they perform their daily activities. The researcher may simply observe, without interacting with the participant, or a session may be more interactive, with participants talking about what they are doing and the researcher asking questions, similar to a contextual inquiry. The goal and primary value of this technique is to observe people’s natural activities.
Covert Observation
Covert observation is similar to naturalistic observation, but the researcher observes people without their knowing that they are being observed. Of course, you can ethically observe people covertly only in public places, where there is no expectation of privacy. For example, you might observe what people do in an airport. The advantage of covert observation is that it eliminates any effects your presence might have on a participant’s behavior. To learn more about this method, see my column, “Becoming a Spy: Covert Naturalistic Observation.”
Participant Observation
Participant observation is a traditional ethnographic method in which the researcher joins a group and participates in their activities. The researcher observes and interacts with group members while performing the same activities. For example, a researcher might become a call-center operator for a few days, with the goal of better understanding such operators’ work and experiences. For more information about this method, see my previous column, “Participatory Observation.”
Differences Between These Methods
Several aspects of observation differ in these various methods of research.
Location of the Observation
In most of these research methods, the researcher visits participants in their natural environment to observe their natural behavior. The exception is usability testing, which can take place at a participant’s location, but often brings participants to a specific testing location such as a lab. Because, in usability testing, all participants perform the same tasks that the researcher has defined, conducting testing in their natural environment isn’t as important.
Amount of Interaction with the Participant
In naturalistic observation, covert observation, and sometimes, shadowing, researchers avoid interacting with participants, so they won’t influence their behavior. Although the presence of a researcher may initially affect participants’ behavior, the researcher attempts to be unobtrusive, so most participants tend to become more comfortable over time.
The advantage of not interacting with participants at all is that you can observe their natural behavior. The disadvantage is that you don’t get to hear their descriptions of what they are doing, and you can’t ask questions in the moment. So it may be more difficult to understand what participants are doing because you have to rely on assumptions—at least until you can ask them questions later.
With techniques that involve much more interaction between the researcher and participant—such as contextual inquiry—it is often much easier to understand what you’re observing. The participants tell you what they’re doing, and you can ask them questions in the moment. The disadvantage, however, is that this interaction can make the situation somewhat artificial. What you see won’t necessarily be what participants would do when a researcher wasn’t present.
Proximity to the Participant
Seeing the details of tasks—such as what participants do with a user interface—requires that you sit very close to them. In such close proximity, it’s almost impossible not to have some type of interaction with participants. So sitting silently and pretending to be unobtrusive would be ridiculous in such a situation. That’s probably why contextual inquiry is the most frequently used research technique on technology projects. Seeing detailed tasks requires close proximity, so not interacting with participants at that distance would be unrealistic.
However, when your goal is to observe tasks that don’t require seeing close-up details or you’re observing the overall daily activities of a single person or group, you can sit farther away and simply observe participants without interrupting them. At a greater distance, there is no social expectation that you maintain a conversation. So you can introduce yourself, then try to be unobtrusive. After some initial awkwardness, participants are likely to relax, and you can observe their natural behavior.
Participants’ Knowledge of Being Observed
In covert observation, participants do not know they’re being observed. This completely eliminates any negative effects of your presence altering participants’ behavior. Again, to be ethical, you can do this only in a public place where there is no expectation of privacy.
In all of the other methods, participants consent to being part of a study, and this can affect their behavior.
Naturalistic Observation
For the remainder of this column, let’s take a closer look at a very useful, yet rarely used method of UX research: naturalistic observation.
Why Use Naturalistic Observation?
Naturalistic observation is in wide use in anthropology and the social sciences because it lets researchers unobtrusively observe natural behavior over long periods of time. Because, in UX research, our timeframes are much shorter and we usually need to view detailed tasks up close, we more often use contextual inquiry, which is a very economical method of learning about people’s tasks. In just an hour or two, participants can walk you through their key tasks.
However, contextual inquiries are somewhat artificial. During a two-hour time slot that you schedule, you ask participants to show you their tasks—regardless of whether they would normally perform those tasks at that particular time. So they might have to recreate or simulate some situations to show you what they would normally do. As a result, you won’t see how those tasks realistically fit into their workday. You might not see examples of certain variations, exceptions, problems, and interruptions that are characteristic of their daily tasks. While you can see what participants do when performing a task on their own, you’re less likely to see the interactions between multiple people that would occur during a typical day.
A naturalistic observation lets you see what happens over a longer period of time, whether you’re observing one person or a group of people. You can see how a normal day unfolds without introducing your own interruptions or influencing participants. For example, while you might hear about particular problems during a contextual inquiry, observing participants over a longer period of time provides a better understanding of how often such problems occur and what causes them.
Use Naturalistic Observation as a Complement to Other Methods
Naturalistic observation is best used as a complement to other activities such as contextual inquiries and interviews. But, since you can’t see up-close details or ask questions, you probably wouldn’t want to use naturalistic observation as your only research method.
Instead, you might first conduct a day or two of contextual inquiries with several participants, then spend the next day or two simply observing the group. The contextual inquiries would give you a good understanding of the participants, their tasks, and the problems and issues to watch for during the naturalistic observation. At the end of the last day, you could conduct additional interviews to get answers to your questions and confirm your understanding of what you’ve observed. By combining these methods, you get the advantages of both methods, while minimizing their disadvantages.
Planning an Observation
As I mentioned earlier, observation involves more than simply going somewhere and passively looking and listening. So planning is necessary to get the best results.
Plan the Goals of the Research
First, decide what you want to learn from the research. If you had all the time in the world, you could begin with a general goal of simply observing whatever happens. But working within a realistic timeframe, you’ll need to focus on specific questions that you want to answer. This shouldn’t mean you won’t be open to noticing other important details. You’ll start with a particular focus, but be flexible and revise your plan as necessary.
Plan Your Research Methods
Decide what additional research methods you’ll use—such as interviews and contextual inquiries—and when you’ll conduct them in relation to the naturalistic observations. Plan how many sessions of each method you’ll conduct and in what order. For example, you may want to conduct interviews or contextual inquiries first, then simply observe participants, and finally, conduct additional interviews to answer any questions that have arisen during observations.
Plan What, Who, Where, and When to Observe
Once you’ve decided which tasks to observe, find out who performs those tasks, where they are, and when they perform those tasks. Since you can’t really stage tasks, as you could when using other research methods, find out where and when you’ll need to visit participants to see the tasks you’re interested in.
Plan Multiple Sessions
If you have time, try to conduct more than one full-day session. Observing participants over several sessions has the following advantages:
You see more variety. Tasks and activities may vary at different times of day, on particular days of the week, or at specific times of each month.
You see the same tasks multiple times. Repetition gives you a better understanding of the tasks, variations between people, and the frequency of activities, issues, and problems.
You can sustain focused attention for only so long. Splitting observations into multiple sessions lets you retain your mental freshness, and you’ll possibly notice different things.
You can use the time between sessions to reflect. Reflecting on what you’ve already observed lets you refine your focus for the remaining sessions.
Conduct Background Research
Since you won’t be able to ask questions during the observation sessions, try to gain a good understanding of what you’ll be observing before you go out into the field. First, interview stakeholders and familiarize yourself with any existing information from past studies, surveys, metrics, and any other sources. Learn as much as you can about the users, their tasks, the tools they use, their environment, and the business domain. This will make it easier to know what details to focus on during observations and help you better understand what you’re seeing.
Scope Out the Location
If possible, check out the location ahead of time, so you can determine the best place to sit. Pick a spot that gives you the best view while also helping you to remain unobtrusive. You may need to choose several locations, depending on what and who you want to observe.
Create an Observation Guide
To simplify notetaking—especially if multiple people are observing sessions and you want to coordinate their efforts—you may want to create an observation guide that lists the types of things observers should focus on. The guide could include events and actions for observers to check off, a map to note where particular actions take place, and what tasks to focus on. Since observation is typically very unpredictable, keep the guide simple. One double-sided page is ideal so observers won’t have to shuffle through several pages. Create something that they can quickly refer to.
Conducting the Observation
Ideally, you should conduct naturalistic observations after you’ve conducted other methods of research such as interviews or contextual inquiries. Not only will you be better informed about what you’ll be observing but participants will also be more comfortable with your presence. They will have met you, learned about what you’re doing, talked with you, and showed you their tasks. At that point, it will feel less strange to have you sitting in the corner watching them all day.
Explain to participants what you’re doing and why you’re observing them. Have them sign consent forms, and ask their permission to record video or audio. Allow participants to ask questions, try to make them feel comfortable, and ask them to go about their normal activities.
Try to Be Unobtrusive
Of course, participants will know that you’re observing them, but try to remain unobtrusive. It usually takes time for people to get used to your presence and overcome the uncomfortable feeling of being watched. However, this doesn’t mean you can’t ever talk with participants. For example, to break the tension, a participant might ask, “How’s it going so far?” It’s all right to have normal conversations in moments like that. But afterward, return to your unobtrusive observation mode.
Observe
Observation involves carefully looking, listening, and thinking about what you’re seeing and hearing, so you can pick out significant details. Based on your planning, you might focus on the following elements:
user characteristics
tasks and the individual steps involved
workflows between people
interactions between people
interruptions
tools, technology, and other artifacts that participants use
sources of information
problems participants encounter
environmental factors such as layout, traffic patterns, where people congregate, temperature, noise levels, and lighting
Or you could use one of various observation frameworks that researchers have developed. For example, AEIOU—which was developed by Rick Robinson, Ilya Prokopoff, John Cain, and Julie Pokorny—stands for particular elements to note during observations:
activities—goal-directed actions, activities, and processes
environments—personal or shared workspaces or common areas
interactions—between people and objects
objects—things people have in their environment and use in their activities
users—the people you’re observing
Try to Be Objective
Although you’ll begin with specific goals, try to keep an open mind and avoid coming in with preconceived notions. However, some bias is unavoidable as researchers try to interpret the sensory information they take in. For example, observers may fall prey to confirmation bias, in which their observations are biased toward confirming what they expected to see.
To help combat such biases, note your observations rather than your interpretations of what you observe. Or, at least, note your interpretations in a separate place or a separate way from your observations. Observations are facts about what you saw or heard—such as the participant frowned. Interpretations are your assumptions about what happened, what people are thinking or feeling, or why people performed certain actions. For example, the participant was confused and angry. Your interpretations might not be correct, so you shouldn’t take them as fact unless you verify them with participants later.
Take Notes
Take rough, shorthand notes instead of trying to capture every detail in complete sentences. Focusing too much on notetaking distracts your attention from what you’re observing. However, make sure you provide enough detail that you can understand your notes later.
After each session, go somewhere else to type up more detailed notes on a computer. Do this while your memory is still fresh from the session. If you wait too long, you may forget things and may not be able to interpret your written notes as easily.
List Your Questions Separately
As questions occur to you, write them down in a separate place from your notes. This makes them easy to find later, instead of their getting lost within pages and pages of notes. Then, when it comes time to ask participants your questions, you’ll be able to locate them easily.
Record the Sessions
If participants consent, make video or audio recordings of the sessions. But don’t rely solely on the recordings rather than taking notes. You may not have time to go back and review the recordings later, but it’s nice to have them as a backup. Even if you don’t have time to review entire recordings, you may want to review specific moments that were significant, confusing, or occurred too quickly to follow.
Take Photos
Take photos unobtrusively using a smartphone, with the flash and shutter sound turned off. Get participants’ permission to take photos at the beginning of a session, but take photos unobtrusively to avoid making participants uncomfortable. If they notice you taking photos, that’s another distracting reminder that you’re observing them.
Schedule the Observation Day Logically
Observation is mentally challenging because it requires careful attention and effort that you can maintain effectively for only so long. Therefore, break up your observations into the following schedule:
Observe for a set period of time, depending on the length of the tasks you need to observe—for example, in 60- to 90-minute segments.
Go to a separate area such as a conference room to type up your written notes in an expanded, more understandable format, while a session is fresh in your memory.
Take a break for at least 15–30 minutes.
Go back to observing again, repeating this process.
Review Your Progress
At the end of each day or between observation days, review your typed notes to gauge your progress and determine whether you need to make any changes to your plan. See whether themes are emerging, you need to shift your focus, you need to observe different tasks or situations, or you need to make any changes to your process or schedule.
Add Naturalistic Observation to Your Process
Most of the user-research methods we use for UX design are somewhat artificial. That’s understandable because we need to sit close enough to participants to see their detailed tasks, we need to hear them explain what they’re doing, and we need to ask them questions. However, naturalistic observation can be a valuable addition to other more common research methods. It provides a more realistic understanding of people, their tasks, and their environment.
Being a researcher, I can connect a lot with your article. Often, observation is the most important factor but doesn’t get the importance in the whole process of research.
I liked your article about the role of observation in user research. You have explained the basics of observation, including different types of observation. It’s important to understand the role of observation in user research. I also liked another article similar to this one: “How to Conduct User Observation.”
Jim has spent most of the 21st Century researching and designing intuitive and satisfying user experiences. As a UX consultant, he has worked on Web sites, mobile apps, intranets, Web applications, software, and business applications for financial, pharmaceutical, medical, entertainment, retail, technology, and government clients. He has a Masters of Science degree in Human-Computer Interaction from DePaul University. Read More
