There’s no denying that, in most cases, in-person research is much better than remote research. But who are we to say that creating good user experiences is more important than the environment? Now, more than ever, it’s important to determine when it’s feasible to save money and the environment by conducting more user research remotely.
Remote user research can be either moderated or unmoderated. In both cases, the participants and researcher are in separate locations. However, in moderated, remote user research, the researcher and the participants go through the research activity together virtually, while in unmoderated, remote user research, the researcher is not involved during the study.
In this column, I’ll discuss the advantages and disadvantages of moderated and unmoderated, remote user research. Then, I’ll reflect on some deciding factors for conducting either in-person or remote user research—or both in combination. Understanding all of these considerations can help you to decide when it’s most appropriate to use in-person or remote methods of user research—and if the latter, whether to do moderated or unmoderated research—or to combine both approaches and get the best of both worlds.
Moderated, Remote User Research
During a moderated, remote user research session, the researcher and a participant typically communicate by phone and use software that lets the researcher observe the participant’s computer screen. In fact, Web-conferencing applications such as WebEx
or software like TechSmith’s UserVue allow the researcher and participant to both see and control each other’s computers. It’s possible to record a user research session, using screen-recording software such as TechSmith’s Morae and a microphone. Using a Webcam to view the participant is also an option, as I’ll explore later.
Advantages of Moderated, Remote User Research
First, let’s consider the advantages of moderated, remote user research.
Eliminating Travel and Saving Time, Money, and the Environment
Remote user research eliminates travel, thereby saving time and money. You can avoid wasted time in airports, there’s no need to rent or reserve a usability lab or other research facility, there’s no wear and tear on your equipment, and usually, incentives can be lower, because participants don’t have to travel to your lab’s location. Obviously, reducing travel offers important environmental benefits, too.
Including Participants from Anywhere in the World Is Easy
If you conduct only in-person user research, budget and time considerations often limit you to using only local participants or traveling to just a few locations. Remote research lets you include people who would otherwise be left out, so you can have a more representative sample. Moreover, it’s often easier to recruit participants when they don’t have to travel to a usability lab to participate.
Conducting Usability Testing Sessions That Are More Representative of Real Situations
In remote usability testing, participants use their own computers in their home or workplace, where they typically feel more comfortable than in a usability lab. In such environments, you can witness participants’ Internet network speed, browser choice, bookmarked sites, and so forth, as well as the natural interruptions and interactions that occur in real-world environments.
Thinking Aloud Is More Natural for Participants
Because you can’t see the participants, it’s even more important than usual for them to think aloud as they work, so you can understand what they are doing. When a participant stops thinking aloud during an in-person testing session, you can usually see why—he or she may be reading, confused, in deep concentration, or frustrated and about to give up. But in a remote session, you miss these visual cues.
Fortunately, because participants can’t see you either, the need to think aloud is obvious, and doing so comes more naturally to participants who are working remotely. Participants are more likely to communicate with you verbally and explicitly narrate their actions as they move through a task, which gives you a better understanding of what they are thinking and doing.
Focusing the Researcher on Observing Rather Than Interacting
Engaging with participants in remote locations removes the dynamics of body language and eye contact, allowing you to focus on your computer screen and your notes. Greater focus can lead to richer insights and more fruitful discussions with participants. Plus, during a remote session, you can log events on your computer, saving time, making data analysis easier, and—as an extra environmental benefit—not using up paper.
Disadvantages of Moderated, Remote User Research
Now, let’s look at the disadvantages of moderated, remote user research.
Seeing Participants and Their Context Isn’t Possible
Since participants’ computer interactions are the focal point of user research, it’s easy to imagine how you could successfully conduct remote usability testing. But remote contextual inquiry is an oxymoron. How can a user interview be contextual when you can’t see the user’s context firsthand?
During a remote user research session, the only context you can see is a participant’s computer screen. You miss seeing the participant’s facial expressions and body language, as well as any interactions with other people, the surroundings, or supplemental materials—for example, Post-it notes, printouts, calendars, or documents.
You can use Webcams to capture participants’ facial expressions and body language. But sending a Webcam to each participant is expensive and coordinating the installation of Webcams and software for multiple participants can be difficult, and seeing participants’ facial expressions through the typically delayed video a Webcam provides is rarely worth the effort. Sending disposable cameras to participants so they can photograph their surroundings and materials is a much less expensive way of filling in some of the details you miss when doing remote user research.
Establishing Rapport with Participants Can Be More Difficult
Distance and the technology for conducting remote research can add a level of impersonality to the interactions between you and participants during test sessions. You’ll have to work harder to establish a sense of rapport with participants.
Maintaining Control Over the Situation Is More Difficult
Because you’re not physically present, you have much less control over a research session. Frequently, coworkers, family members, phone calls, instant messages, and even pets interrupt participants. Although these interruptions provide a sense of participants’ real lives, which is part of what you want in a contextual inquiry, they can completely disrupt tasks during usability testing and ruin your studies’ results.
Participants May Invite Others to Attend
For some reason, participants in remote user research occasionally invite other people to attend a test session—often a coworker they believe might have useful information. Perhaps the use of a conference call and Web-conferencing software causes participants to think of a user research session as a meeting. If you’re lucky, the participant will inform you that the extra person is present. However, your inability to see who else might be lurking in the background makes it difficult to eliminate the presence of these extra attendees.
Web-Conferencing Software Can Have Technical Problems
It happens less often than it used to, but some participants still have problems using Web-conferencing software. Those with a slow Internet connection may not be able to connect and share their screen. Slow connections can also delay your view of a participant’s actions.
