Doing User Research Faster and Cheaper
Published: May 3, 2010
Despite our seeing some initial signs of a recovery, for most people the economy still sucks. Companies have less money to spend and are more cautious about how they spend it. Companies that haven’t already cut user research from their project plans altogether are asking researchers to achieve the same results for less money, in less time—or just to do less. Is it possible to scale back user research and still provide value? If so, how can we do things faster and cheaper?
An inherent problem with traditional user research is that it tends to require a lot of time. Any activity that involves meeting with multiple people requires the time-consuming steps of recruiting, coordinating schedules, perhaps traveling, the actual session time, and the downtime between sessions. User research generates a great deal of qualitative and quantitative data that researchers must consolidate, type up, analyze, and report clearly to project team members or clients. The larger the scope of the user research, the longer it takes to provide thorough, detailed, articulate results.
Clearly, some information about users is better than none. In this column, I’ll review a variety of ways in which you can scale back the time it takes to do user research, while still providing valuable results.
Begin with an Expert Review
With the right combination of expert review and usability research, you can make the most of a limited research budget. Expert review techniques such as heuristic evaluation, cognitive walkthrough, and accessibility evaluation are quicker and less expensive ways of finding user interface problems. Obviously, these techniques are far less time consuming and expensive than methods that involve users. Expert review complements rather than replaces usability research by identifying the obvious problems with a user interface. Applying best practices and experience, a skilled usability professional can find and eliminate many usability and design problems, saving your user research budget for questions that only actual users can answer.
Use Existing Data
For Web sites, you may already have access to a wealth of information your company has gathered, including site metrics, search logs, and other tools that record visitor’s response to its user experience. Newer tools like Click Tale, Tea Leaf, Crazy Egg, and others provide a variety of visualizations of user behavior, including click heat maps, attention heat maps, and the ability to replay user sessions on a Web site. Although the data from these tools is somewhat limited and can be difficult to interpret, it can help point you toward usability issues that need further investigation through user research. Starting with this existing data gives you a headstart in narrowing the focus of your research.
Break Large Research Projects into a Series of Smaller Ones
If a large-scale user research project is financially out of reach, re-imagine it as a series of smaller, less expensive projects. Properly conceptualized and sequenced, these small projects can bring immediate value and, over time, still lead you to the larger understanding you need. This technique works especially well for teams that are continually improving a user interface, whether for a Web site or an application. For those who are paying for the research, it can seem less risky to commit to smaller projects, one at a time, rather than committing to one large project.
Limit the Scope and Shorten the Sessions
As user researchers, we are naturally curious and prefer to do exhaustive research. We want to get as much as we can out of every session. However, trying to pack as much as you can into your research sessions can add exponentially to the amount of data you’ll need to consolidate and analyze. With time and budget limitations, it’s better to narrow your focus to the most important information you need to gather. The tighter your focus, the less time your study and analysis will require.
Limiting scope also allows you to have shorter sessions, and you’ll end up with fewer notes to type up and analyze, which lets you turn around the results much more quickly.
Do Your Own Recruiting
Save money by recruiting participants from your Web site, screening them with a questionnaire, and saving their information in a participant database. Intercept site visitors with an invitation to participate in future usability studies. Your site can immediately take those who express interest to a screening questionnaire that saves their responses to your database. Over time, you can build up quite a large pool of potential participants on which you can draw, without incurring the expense of using a recruiting company.
Include Fewer Participants
Cut down the number of participants in each study. You may have to settle for fewer participants than you’d like, but some research is better than none. For example, you could do guerilla usability testing, conducting iterative rounds of usability testing with five participants in each round. The advantage of limiting the number of participants in each round is that you can conduct more iterations of research and continually build upon the knowledge you’ve gained previously.
Focus on the Primary Audience
You may think you need to do studies involving multiple user groups. However, including even a small number of participants from each user group can result in a large total number of participants. Recruiting, session time, and analysis increase exponentially. Instead, identify and focus on the primary user group first, assuming you’ll design mainly for that audience. As time and money permit, add participants from secondary user groups.
Test Early with Low-Fidelity Prototypes
Conduct usability testing early in the design process with paper prototypes. Not only do low-fidelity prototypes take less time to produce, but testing is also more informal, and you can easily conduct tests anywhere. Early user feedback lets you make quick changes, then test the next iteration. The speed and informality of this method may enable you to fit in more rounds of iterative testing.
Skip the Usability Lab
If you don’t already have a usability lab, don’t spend money renting one. Instead, you can conduct testing in realistic contexts like your participants’ homes or offices. Recording equipment such as a video camera, audio recorder, or still camera can be helpful, but is not absolutely necessary. All you really need is a pen and paper to take notes. Testing on site is easier for your participants, because they don’t have to travel; so you can save money on incentives, too.
Conduct Remote User Research
You can conduct usability testing, interviews, and even contextual inquiries remotely, using your phone and Web conferencing software that lets you see participants’ screens. This lets you easily reach participants anywhere in the world, while saving travel time and costs. Again, since participants do not have to come to you, incentives can be lower. And since participants are not in the same room with you, you can save time by typing your notes on a laptop during a session, logging issues yourself instead of requiring another person to do the logging.
Conduct Unmoderated, Remote User Research
Using unmoderated, remote tools for usability testing, card sorting, tree testing, surveys, and other types of online research can save a lot of time and money. Participants complete these studies in their own time, and the tools automatically collect and present data in ready-to-use results. This eliminates time-consuming tasks, including scheduling participants, moderating the sessions, recording the data, and preparing the data for analysis. You’ll be able to spend more of your time designing the study and analyzing the results.
However, unmoderated research tools are somewhat limited and are not well suited for eliciting in-depth, qualitative information. Be careful to use them when they can provide value, not as a substitute for any and all user research. For a more thorough analysis of the advantages and disadvantages of unmoderated research tools, see my previous column, “Sustainable User Research.”
Self-reporting methods such as diary studies provide other less expensive, unmoderated user research options. In diary studies, participants record their own experiences with a particular activity, user interface, or product over time, in paper or online journals. This is an excellent way to investigate how experiences unfold over time.
Conduct Group Sessions
The term group sessions might call focus groups to mind—along with all of their limitations—but there are other useful ways of working with or gathering feedback from groups of people. Consider activities like affinity diagramming, group card sorting, participatory design, and other creative methods of getting insights from a group. Group sessions require less time than individual sessions, because they typically produce less data that you’ll need to consolidate and analyze. Although getting less data can save you time, remember that it may also provide fewer insights than individual sessions would.
Don’t Type Up Your Notes
If you really need to save time, it’s worth considering some very basic shortcuts that won’t compromise your goals. For example, typing up notes from each session takes a great deal of time and produces lots of information you’ll need to analyze. But you might not really need all of that information to answer your research questions—and even if you do, you might not need to type it all up. Practice working from fewer typed notes and keep more of your findings at a high level.
Although you may feel that you need to type up your notes just to remember everything, you’ll be surprised by how much you can recall off the top of your head. To work in this way, use your test plan or interview questions as a guide and type up your overall findings for each question from memory after the last session rather than by referring to your notes. But what about those important or subtle details you’ll miss by not transcribing and analyzing everything you noted? If you must, go back to your informal notes for a quick skim to reassure yourself. In the end, though, if your time really is limited, so is the amount of detail you can provide to your team or client.
Produce Higher-Level, More Informal Deliverables
Provide a high-level, summary report or presentation, offering highlights of your findings, instead of a detailed report. This could range from bullet points in an email message, to annotated screenshots, a brief report, or a summary presentation. If you’re really pressed for time and budget, you can sometimes cut out creating a deliverable entirely. Depending on your company’s or client’s goals, an informal discussion of the findings and recommended design changes can take the place of a deliverable.
In an ideal world, we would have all the time we needed to conduct extensive, detailed user research. But even in the best economy, we rarely have that luxury. Instead, we can look at the current economic situation as an opportunity to review our methods and determine how to maximize the value of user research, while minimizing both the time it takes and its cost. When the economy eventually improves, the things we’ve learned during these lean times can help us provide more effective and efficient user research.