What’s the Difference Between Moderated and Unmoderated User Research?
In moderated studies, the user researcher and participants meet either in person or remotely via phone and Web-conferencing software. Study sessions typically last an hour or more. Traditional usability testing, field studies, user interviews, and focus groups are examples of moderated studies.
In unmoderated, remote research, the user researcher and participants never meet. Participants complete a study at their own location, on their own time. An online tool collects data and presents the results to the researcher. Sessions typically require only a fraction of the time moderated research requires. Examples of unmoderated, remote research are online card-sorting studies, surveys, and online usability testing.
How Is Recruiting for Unmoderated, Remote User Research Different?
Because participants complete unmoderated studies on their own time and an online tool automatically captures, then presents the data to the researcher, it’s easy to include hundreds or even thousands of participants without investing the time and effort moderated sessions require. The main challenges are finding a very large number of potential participants, ensuring that they are representative, and convincing them to participate. This requires very different recruiting methods than those we typically use for traditional, moderated user research.
Three Methods of Recruiting Participants
There are three ways you can invite people to participate in an unmoderated study. You can
- email a link to your study to potential participants
- use a recruiting company’s online panel of participants
- use online intercepts
The appropriate method to choose depends on the specifics of your study and the type of people you are trying to recruit.
Emailing Potential Participants
Emailing a link to your study is an effective method if you can easily identify potential participants and obtain their email addresses. However, emailing a large number of people can be time consuming.
Gathering a List of Potential Participants
Existing lists of email addresses—such as customer lists, membership lists, or employee directories—are good places to gather a list of potential participants. Ideally, these lists offer enough information so you can narrow them down to the type of participants you want. Even after narrowing the list, you may need to do some additional screening.
Some unmoderated, remote user research tools can present screening questions at the beginning of a study and reject respondents who don’t fit your profile. The risk here is that each question adds to the length of the study, increasing the chance that people will abandon it. Be careful to include only screening questions that are absolutely necessary.
Another way to build a list of potential participants is to use an intercept questionnaire that lets you recruit visitors from your Web site. With screening tools like Ethnio, you can set up an intercept on your site to invite visitors to participate in future studies. Those who want to volunteer must take a screening questionnaire, then the tool adds those who qualify to a database of participants you can use when emailing invitations for future studies.
Determining How Many People to Invite to Participate
The ideal number of participants depends on your research goals. You should determine the minimum number of participants you need for acceptable results and, if necessary, a maximum number. The minimum helps you know when you’ve received enough responses to end the study. The maximum is also helpful, because some tools allow you to automatically end a study when you reach a set limit. This can prevent you from paying too much in incentives.
Because mass emailing of invitations to studies often results in low completion rates, you should send email messages to far more people than a study requires. Some of the messages will get blocked by spam filters, only some recipients will open and read them, only some of those who read them will click the link to the study, and fewer still will actually complete the study. Response rates are often as low as five percent. So it’s best to assume a low response rate and, therefore, invite an appropriately large number of potential participants. For example, if you want 100 participants to complete a study, invite 2,000 people—assuming a five-percent response rate.
Creating an Effective Email Message
Craft a short, effective email message that describes your study, highlights the incentive, and provides a link to the study. To keep your message brief, don’t include detailed instructions about how to complete the study. Instead, save those details for the instructions at the beginning of your online study.
Establishing Credibility
Your email message must quickly establish credibility and trust with potential participants. Understandably, people are wary of clicking links in messages from people they don’t know. Although mass emailing requires less work, sending messages to individuals is more effective in establishing trust and, therefore, it is less likely people will ignore your messages.
Personalize your email messages by
- sending each individual person a message, instead of sending one message to many email addresses
- using the recipient’s name at the beginning of your message
- writing in a friendly, professional tone
- describing who you are and who you’re representing
- including your name, title, company name, company Web site link, and contact information at the bottom of your message
Who an email message comes from is very important in establishing credibility. Because a message from a familiar source gets more responses, it’s a good idea to have a client for whom you’re doing user research send the email messages. For example, employees are more likely to trust an internal email message from a fellow employee, and customers are more likely to trust a request from a company they patronize than a request from an unknown third party.
If you must send the email messages yourself, mention the name of someone with whom the recipient is familiar at the beginning of the message. If you are recruiting employees, providing your client’s name and contact information is the next best thing to having your client send the messages directly. Although few recipients will actually contact you or your client, giving them the ability to do so increases their trust in you and the likelihood they will respond positively.
