Scoping User Research

Involve UX Researchers in Scoping

While it might seem obvious that UX researchers are the most qualified people to scope a project’s user-research activities and estimate the time they’ll involve, teams do not always involve researchers in scoping. Within consultancies, salespeople and managers often scope user-research projects instead.

The problem with salespeople scoping user-research projects is that they don’t have enough knowledge of user-research methods, so they’ll either sell only the approaches with which they’re familiar or that they think a client would want to purchase. For example, at one company where I worked, the salespeople got stuck in a rut, always selling clients the same research method—contextual inquiries—with a standard number of participants, and over a standard period of time, regardless of the needs of a particular project.

The problem with managers scoping projects is that, while they may have conducted user research themselves in the past, they probably won’t be conducting the research for the project they’re scoping. If they’ve been removed from the day-to-day realities of conducting user research for some time, they might not know the latest methods, might be unaware of the time and effort specific methods require, or might be overly optimistic in their estimates of the hours they would require—perhaps because it’s more likely that a client would sign a contract for a lower-cost project.

So, instead of relying only on salespeople and managers, include UX researchers in the scoping process. UX researchers know the right questions to ask, can choose the most appropriate methods, and can better estimate the time necessary. With the researchers’ input, the manager can then create the scoping plan, but have a researcher review it to make sure it’s accurate.

Add More Time for Complexity

The more complicated and unfamiliar a project’s subject matter and domain, the longer user research takes. When scoping user research for such projects, include additional time for stakeholder interviews or workshops. Extra stakeholder research helps the UX researchers learn more about the domain and business needs and provides an overview of user groups and their tasks. Spending this extra time on learning makes it easier for UX researchers to understand what they observe during the user-research sessions.

Determine How Many Research Sessions to Conduct

For qualitative user research, there is no magic number of participants to include. You need to see enough participants performing the same tasks to be able to observe patterns and differences. The total number of participants and the length of the research sessions determine how long research and analysis take.

Answer the following questions to help you assess how many research sessions to conduct.

How Many User Groups Are There?

How many different types of users are there? Which user groups should you include in your research? You may not need to include every user group—especially if you have time and cost limitations. For example, if you were redesigning a healthcare application, you might focus your research on nurses and doctors, skipping hospital executives and IT staff users.

How Many Tasks Do You Need to Observe?

Which tasks do you need to understand? Most likely, you won’t need to observe everything users do—only those tasks relating to the scope of your project. Of course, sometimes you won’t know what tasks it’s important to observe until you begin the research sessions. Some of the tasks you expected to observe may no longer be relevant, and you may learn about additional tasks it’s important to observe.

How Many Times Should You See Each Task Performed?

Again, there is no magic number of times you’ll need to observe each task. Ideally, you should see enough repetition to understand the tasks and observe common patterns and variations across participants. If the subject matter is especially complex or unfamiliar, include extra sessions to ensure that you really understand what you’re observing.

How Long Should the Research Sessions Be?

Once you know which tasks you need to observe for each participant, determine how long it typically takes to perform each task. Consider that tasks can take significantly longer when participants are demonstrating them and talking you through what they’re doing—as in a contextual inquiry. Also, factor in the time it takes to introduce your research method at the beginning of each session, as well as time you may need for asking additional questions that arise during a session.

In addition to considering the amount of time you would ideally want, consider the amount of time participants would agree to give you. This depends on how much time they can spare, how eager they are to help, the incentive you’re giving them, and the physical and cognitive demands of the session. For example, high-level executives may be able—or willing—to give you only 30 minutes to an hour of their time, while managers may simply inform lower-level employees that they must participate in a two-hour research session. Two to three hours is likely the practical limit for most people before they become tired.

If there are too many tasks or the tasks take too long to fit all of the tasks into a single session, you can either schedule multiple sessions with each participant or split up the tasks among several different participants. For example, if you need to observe five long tasks, but you can only do about three tasks per session, you could observe tasks A, B, and C with five participants and tasks D and E with five other participants.

How Difficult Is It to Recruit Participants?

Recruiting and scheduling participants can require a major expenditure of time and effort. If you don’t already have an easy source of willing participants, recruiting can take a few weeks. In fact, recruiting difficulties can delay your project. So be sure to assess whether you’ll have any problems finding and recruiting the number and types of participants you need.

Does anyone on the project already have lists of potential participants—for example, customer, member, or employee lists? Might you need to use a recruiting company to find enough participants? If you do need to use a recruiting company, get estimates from several companies. How easy or difficult do they think it would be to find the types of people you need?

Once you’ve identified enough potential participants, how easy or difficult would it be to convince them to participate? Will they already have incentives to participate—such as a willingness to help you improve a product they use, altruism, curiosity, or pressure from management? Would you also be able to provide sufficient financial incentives to get them to participate?

If you anticipate any of these difficulties, plan extra time for recruiting and scheduling participants.

Determine the Location for Your User Research

Ideally, user research involves going to participants to observe them performing their tasks in their usual context. However, depending on their location, traveling to participants can add a lot of extra time and expense to a research project. Of course, remote user research is possible in some situations and can save a lot of time and money. To determine the best solution for your project, answer the following questions.

Where Are Participants Located?

Initially, assume that you’d travel to visit participants and answer these questions:

• Does the geographic location of participants make a significant difference in their behavior or tasks? If not, it might not be necessary to include participants from different locations.
• Would local or overnight travel be necessary?
• Are the participants’ locations distributed widely, requiring a separate trip to visit each participant? Or are they grouped together so you can travel to just a few locations and meet with several participants in each location?
• Can you split up the work among UX researchers in various locations? If there are researchers in other cities, perhaps each researcher can meet with participants in their own area to save on travel.

Will You Be Able to Visit Participants?

Consider whether participants would want you to visit them in person at their location as well. For certain activities, they might not want anyone to come into their environment and observe them.

In some cases, you might not be allowed to visit participants on site. If your participants are employees of a client company, you’ll need to get their manager’s permission to visit them. In some situations, there may be safety, security, or privacy concerns that would prevent your visiting employees in person. Plus, some companies have restrictions on the use of recording devices and photography.

Can You Conduct Research Remotely?

For remote user research, participants share their screen during an online meeting, While remote user research is not ideal, it can be an acceptable alternative for some projects. Remote sessions save a lot of time and money, but you can’t directly observe participants’ tasks and environments. However, if most of your participants’ tasks take place primarily on a computer, you can observe most of the context of their tasks.

Determine the Length of the User-Research Phase

While every project is different, there are some basic guidelines that can help you to determine the length of the various parts of your user-research phase.

Planning

Provide at least a week for planning the user research. This time includes getting up to speed on whatever a company currently knows about users and their tasks, planning the research, creating a discussion guide, and reviewing the research plan with the clients and project team. Include at least an additional week if you need to conduct stakeholder workshops and interviews.

Recruiting and Scheduling Participants

Unless participants are readily available and willing to participate, include at least two weeks for recruiting and scheduling participants. This time can overlap the planning week. While two weeks may sound like a long time, recruiting involves a lot of back-and-forth communication for scheduling, which can take some time to coordinate.

Conducting the Research Sessions

The overall duration of a research study depends on the number of sessions, the length of each session, and travel time. A manageable day of research sessions can accommodate approximately five one-hour sessions, three two-hour sessions, or two three-hour sessions.

Also, consider the time between sessions—for traveling between participants, taking breaks, and meals. Since sessions sometimes start late or go longer than you’ve planned, ensure that there is a buffer of at least 30 minutes to an hour between sessions. If one or more recruiters is doing the recruiting and scheduling for you, give them a schedule to fill—or at least share these guidelines with them—to prevent their overbooking your days.

Analysis

The time analysis requires should correlate to the number of hours you’ve spent on research sessions. Some UX researchers have suggested a ratio of three hours of analysis for every hour of research. Therefore, if you’ve conducted ten two-hour sessions, or 20 hours of research, that would require 60 hours of analysis. Of course, you must balance analysis time with legitimate concerns about time, money, and a team’s impatience to get the results of a study, but try to avoid shortchanging the analysis phase. It’s foolish to spend a lot of time and money conducting research, then rush through the analysis phase, which always yields poor results.

Creating Your Deliverable

Make sure to include a separate task in the project plan for creating your user-research deliverable. If you were to lump the time this takes into the analysis task, it would squeeze your analysis time. Although some people have downplayed the importance of creating deliverables in recent years, it’s almost impossible to convey research findings adequately without some kind of deliverable. This doesn’t need to be a long, formal report, but a research deliverable is necessary to convey the key findings to those who weren’t able to participate closely in the research. Plus, a deliverable lives on as a record of the research, whose findings you can apply to future efforts.

The Other Approach to Scoping User Research

What I’ve described in this article is an ideal approach to scoping the research phase of a project. This approach applies if your client is open to determining the ideal time necessary for conducting user research. However, the resulting time and cost estimates might not fit into their budget, so you may need to make some adjustments.

The other approach to scoping user research occurs when your client says, “This is how much time we have. What can you fit in that amount of time?” In such a case, your only option is to try to fit as much of the ideal as possible into what is doable within your client’s limited timeframe and budget. There are always shortcuts you can take. However, when a timeframe for user research is impossibly short, it might actually be better not to do any user research up front rather than rush through activities that aren’t really user research at all.