The Moderator’s Survival Guide: Handling Common, Tricky, and Sticky Situations in User Research

By Donna Tedesco and Fiona Tranquada

Published: April 7, 2014

This is a sample chapter from Donna Tedesco and Fiona Tranquada’s new book, The Moderator’s Survival Guide: Handling Common, Tricky, and Sticky Situations in User Research. 2014 Morgan Kaufmann.

Moderator's Survival Guide

Chapter 2: In the Trenches: Six Steps for Handling Situations

Once you’ve understood your role as the moderator and the different styles of moderating, you can start thinking about how to handle the different kinds of situations that may arise during the session. Some of these situations will challenge your interpersonal skills; others will push the limits of your troubleshooting and crisis-handling capabilities. Although there are common circumstances you’re almost guaranteed to encounter, we can’t predict what else might happen. Even after our years in user research, we’re still surprised—and sometimes delighted, shocked, or horrified—by what happens when we’re one-on-one with a participant. This chapter provides a set of guidelines (as shown in Table 2.1) to help you decide exactly what to do when something unexpected, sticky, or tricky happens during your session.

Table 2.1—Step-by-Step Guide for Handling Any Situation

  1. Take a moment to evaluate the situation before jumping to action.
  2. Resolve any threats to physical safety.
  3. Verify that you’re not causing or magnifying the situation.
  4. Check the participant’s comfort level.
  5. Use careful language and tone to probe on the situation and begin to resolve it.
  6. Regain control to bring the session back on track.

2.1 Take a moment to evaluate the situation before jumping to action

“You need to make sure that you understand the cause of the situation so you can address it appropriately. Rushing to do or say something may lead you to inadvertently make the situation worse or further compromise the results of the session.”

You’re running an interview with a remote participant over the phone. The session seems to be going well until the participant doesn’t respond as quickly to your questions. You hear him whisper to someone in the background, but can’t understand what he says. What, if anything, should you do?

When you encounter anything outside of expected session circumstances, take a moment to step back and evaluate the situation. As the moderator, you keep track of so many things that it can be difficult to take that time. But doing so is vitally important. Why?

  • You need to make sure that you understand the cause of the situation so you can address it appropriately. Rushing to do or say something may lead you to inadvertently make the situation worse or further compromise the results of the session.
  • The situation may resolve itself. In this remote session example, a few moments may be enough for the participant’s whispering to stop and for you to be able to move on with your questions. Also, consider the example of a usability study participant who becomes confused or frustrated. A delay of a few extra seconds is often enough time for a participant to find his bearings and continue without any intervention from you. This delay may also be all a distracted participant needs to reengage with a task or question on his own.
  • If the situation persists, those additional seconds will keep the participant from seeing you sweat—or, in a remote session, hearing you fumble your words as you try to figure out what to do. Remember that, as the moderator, you establish the pace for the session. Take this moment to think through your options and prepare what you’re going to do and say if the situation continues.

This moment may feel like a lifetime to you, but it doesn’t necessarily feel that way to the participant! Even in a potentially life-threatening situation—for example, a fire alarm goes off—take a moment to gather as much information as possible before proceeding. Your need to respond appropriately should outweigh your instinct to react immediately. Remember that you’re responsible for the participant’s well-being as well as your own.

“Think of this moment as securing your own oxygen mask before assisting other passengers.”

If the situation is especially tense or you’re not quite sure what’s going on, you may want to take your moment in another room, away from the participant. Taking a break is an incredibly powerful tool that we’ll discuss more in section 3.7. A break gives you a chance to think about what’s happening in the session without the pressure of monitoring everything else. If you have observers, you can use this break to check with them and get a “gut-check” on what you think is happening and why. The participant may also benefit from a few minutes to cool off, or deal with his distractions.

Table 2.2 shows examples of what you can say if you need to take a break. Note that you can use these phrases even if they aren’t entirely true—consider them mild deceptions for the greater good. Pretexts are discussed further in the sidebar “Using a Pretext.”

Table 2.2—What to Say: Taking a Break

For a usability study with a multiroom setup:

  • “We’re at a good stopping point right now. Let’s take a short break and resume the session in five minutes.” (If appropriate given your setup, you can add: “Feel free to leave the room to get a drink or go to the restroom while I’m gone.”)

For on-site and remote sessions:

  • “I’m so sorry, but would you mind if I took a quick five-minute break to get a drink?” or “This seems like a good stopping point. Why don’t we take a quick five-minute break?”

USING A PRETEXT

While you should avoid misleading the participant, there are times when the truth is going to be harmful. For example, if you notice that the participant does not have the characteristics you need and you want to consult with your stakeholders—who are in another room—about what to do, you should not express your concern about his qualifications directly to the participant. Being honest may hurt his feelings or make him nervous about participating. Instead, make an excuse to leave the room.

Technology can be blamed for many things and provides a convenient excuse if you need to leave the room or end the session early. Just make sure that you have something realistic to blame so that your reason for leaving isn’t obvious. For example, if you say, “The computer isn’t working correctly,” the participant may realize that you’re lying because he can see that it actually is working. But if you say, “Our prototype isn’t working the way it’s supposed to,” he does’t have any reason to know that it is just an excuse.

Examples of what you might say include:

  • “It seems like we may be having technical issues with our recording equipment. Let’s take a short five-minute break while I run next door to look into it.”
  • “I apologize, but I just noticed that there is something wrong with our prototype. Please excuse me for a few minutes while I go to the other room and see if anyone from our team knows what’s going on.”
  • “I’m sorry, but it looks like the product I wanted to get your feedback on isn’t working. I have a few more questions for you, but after that, we’ll end the session early. You’ll still get your full compensation though!”

If you don’t feel comfortable blaming the technology or it isn’t applicable—for example, in an interview where you earn’t using any equipment—you can use the study itself as a pretext:

  • “I meant to give you your compensation at the start of the session. Let me actually go grab that now. Let’s take a break for a few minutes.”
  • “I just noticed that I’m missing a page of my questions. Let’s take a quick break so I can run next door and print that out.”
  • “The rest of these tasks/questions earn’t applicable to you/your role, so we’re going to skip them. This means we may be done a little bit earlier than expected, but don’t worry, you’ll still receive your full compensation!”

2.2 Resolve any threats to physical safety

“The participant’s perception of his safety is just as important, even if you think he is safe. For example, we’ve heard of moderators continuing a session through a small earthquake or a fire drill without saying anything to the participant. Be sensitive to the participant’s comfort and keep him informed.”

In the middle of a contextual inquiry, you hear a low rumbling sound. You take a moment to evaluate the situation, and quickly realize that you’re experiencing an earthquake. What do you do next? (For more about how to handle earthquakes and other natural disasters, see section 14.2.)

When a situation involves a physical threat, you need to respond to and address the threat as soon as you’ve taken a couple of seconds to understand what’s going on. Respond quickly and appropriately, even if that means that you’re unable to continue the session. The safety of you and the participant takes priority over your data collection needs!

Keep in mind that the participant may not know what to do, especially if he is in your facility. You must take responsibility for his safety as well as your own. For example, if a fire alarm goes off or your building is evacuated, bring the participant with you as you follow standard evacuation procedures. While the participant may end up seeing your observers, your top priority is to keep him from physical harm instead of maintaining the integrity of the session.

In the example of an earthquake, you should instruct the participant about where to go—for example, under a desk or in a doorway. If you’re moderating a usability study from an adjacent control room, and you think it’s safe to spend another couple of seconds using the microphone, tell the participant that you’re experiencing an earthquake and instruct him to go under the desk or in the doorway immediately.

If you’re in the participant’s space, follow his lead to exit the building safely or follow the appropriate safety protocol for the event that you’re experiencing.

Depending on the severity of the situation and safety of the environment, you may want to cancel or end the session rather than asking the participant to wait around until the situation is resolved. If you do this, be sure to give him his full compensation and thank him for the feedback provided. If the environment is safe, and the participant is not visibly shaken and willing to continue, offer to take a short break before resuming so you and the participant can both regain focus.

2.3 Verify that you’re not causing or magnifying the situation

“If the participant is remote, you might need to provide acknowledgments more frequently to reassure him that you’re still there and paying attention.”

During an interview, the participant starts making snide remarks after each question you ask. You’re not quite sure why he is responding that way. The issue does’t resolve itself, and taking a break does’t help. There is no physical danger to resolve but you’re worried about the participant’s attitude and its effect on the feedback he is providing. What, if anything, should you do?

Let’s step into some awkward territory for a minute, but we promise, it’s for your own good! Sometimes a situation is magnified, if not created, by something that you’re doing. Part of being a good moderator is being humble enough to constantly evaluate and reflect on how you’re interacting with participants. Before you jump to a conclusion about why the participant is behaving in a certain way, try to detect anything you may have done that led to this behavior.

Table 2.3 shows some examples of problematic moderator behavior that we’ve observed, the moderating styles (as identified by Table 1.1) that may encourage this behavior, and the effect it can have on the participant’s comfort level.

Table 2.3—Examples of Problematic Moderator Behavior and Its Effects

Moderator Behavior (Moderating Style) How the Participant May React

Over-the-top fake (Friendly Face)

You’re trying to be very kind and respectful toward the participant while maintaining a friendly tone. However, you don’t realize that you come across as patronizing or disingenuous.

The participant becomes irritated and curt with his feedback. He may feel like you’re not taking his responses seriously.

Buddy-buddy (Friendly Face)

You want the participant to be comfortable, so you start acting like the two of you are old pals. You laugh together and find yourself making jokes. He tries to make you laugh and looks to you for affirmation, as with any natural friendly exchange. For an example of what this may look like, see Figure 2.1.

The participant begins to try to please you, unknowingly reflecting your attitudes and biases. His efforts to please you throw him off his natural course of interaction and keep him from answering questions fully and honestly.

Obliviously abrasive (Down to Business)

You maintain a serious, expectant tone as you ask questions. However, your tone comes across as overly stern. You also stare intently at the participant without looking away or blinking while he talks. For an example of what this may look like, see Figure 2.1.

If the participant is naturally talkative and high profile, he may feel insulted or put off by your serious tone. If he started out nervous, your tone may only increase his discomfort. The intent staring may unnerve him to the point of asking to leave.

Death by silence (Down to Business; By the Book)

You’re trying to be as neutral as possible so you take a very minimalistic approach to moderating. You’re completely silent most of the time, using only brusque probes—for example, “And…?”—even though the participant is obviously nervous and uncomfortable with the silence.

The participant starts to blame himself and apologizes for “doing a bad job.” Because you don’t respond, he becomes more uncomfortable.

Clock watcher (By the Book)

class="last-row"

You’re trying to stay on schedule, and look at the clock or your watch at the most inopportune times. You ask a question from your study plan and, while the participant answers, look at the clock, look ahead in the study plan, or obviously pay no attention to the participant’s answer.

The participant gets frustrated and makes sarcastic or snide remarks because he does’t feel like you’re listening to him.

As discussed in Chapter 1, there is a delicate balance between the goals and method of your study, the type of participant you have, his current emotional state, and the appropriate moderating style to use. These last two elements in particular can change multiple times throughout the session. You should always carefully monitor how the participant responds to avoid making him uncomfortable. Again, part of the art of moderating is in evaluating your technique throughout the session and adapting it as necessary.

Figure 2.1—Examples of moderating behaviors—the Obliviously abrasive and the Buddy-buddy

Examples of moderating behaviors—the Obliviously abrasive and the Buddy-buddy

© 2013 Mark Ainscow, used with permission.

LEARNING TO ENJOY THE SILENCE

Most moderators are uncomfortable with silence during a study, especially new moderators! You may feel especially uncomfortable if you’re using a think-aloud protocol, since it may seem like the participant forgot your instruction to share what he is looking for or expects to see.

However, silence can be a valuable tool. Sometimes allowing a participant to quietly process and then answer the question or resume thinking aloud is more valuable than probing immediately for his thoughts whenever he becomes silent. For example, most people have difficulty thinking aloud while reading thick blocks of web content, so hold off on your questions while the participant is obviously reading. Similarly, not answering the participant’s question right away can lead the participant to answer it on his own. However, if he shifts toward you, or repeats the question while raising his voice, it’s time to say something.

It takes practice to become comfortable with this approach, but it pays off when the participant does’t feel rushed or harassed by your frequent reminders to talk about what he is thinking.

To find out if you’re creating or contributing to a problem:

  1. Check your body language:
  • Are you too close to or too far from the participant?
  • Are you maintaining eye contact, or are you making too much eye contact?
  • Have you been yawning or glancing at your watch?
  • Are you sitting upright, or are you slouched down in your chair?
  • Have you been frowning or grimacing instead of maintaining a friendly, neutral expression?
  1. Replay what you’ve said during the session:
  • Have you neglected to reassure the participant that you’re listening to him? Have you been chiming in with the occasional “mm-hm” to encourage him to continue talking?
  • Have you been talking too much, and not letting the participant talk?
  • If running a usability study with a think-aloud protocol, did you give the participant enough time to respond before reminding him to think aloud? (See the sidebar “Learning to Enjoy the Silence” for more on this.)
  • Did you stop using the participant’s language and start using language from the product’s user interface, domain, or any other jargon he may not understand?
  • Are you using close-ended or leading questions that “give away” the answer you want or expect to hear?
  • Have you been laughing with or making jokes with the participant?
  1. Replay your paralanguage—how you said what you said:
  • Did you sound judgmental about an opinion or detail that the participant shared?
  • Did you insinuate that the participant did something incorrectly?
  • Could your tone be heard as patronizing or disingenuous?
“Participants are quick to notice anything different, so if you suddenly started writing when you hadn’t done so previously, he may feel that he is doing something wrong.”
  1. Evaluate your note-taking behavior:
  • Are you shuffling papers while the participant attempts tasks or answers your questions?
  • Have you been taking notes only when the participant says or does something “wrong” or unexpected?
  1. Check the physical setup of the room:
  • Is the temperature too hot or too cold?
  • Is there anything in your equipment setup that the participant seems to be struggling with—for example, a monitor that is pushed too far back on the table, or a chair that is set too high or too low for the participant?

We’ll talk more about how to build awareness of your own behaviors while moderating in Chapter 16.

2.4 Check the participant’s comfort level

“Once you’ve verified that the situation you’re seeing hasn’t been caused or exacerbated by your own behavior, check to see if the participant’s degree of comfort has declined to an unacceptable level.”

Once you’ve verified that the situation you’re seeing hasn’t been caused or exacerbated by your own behavior, check to see if the participant’s degree of comfort has declined to an unacceptable level.

Most user research starts out being somewhat uncomfortable for the participant. He may arrive without knowing exactly what he’ll be asked to do, or he may worry that he is being tested in some way. If you are coming in to his space, he’ll be unable to get away from the typical hassles of his environment, which may become magnified by the uncertainty of what he’ll be asked to do during the session.

Unless you’re doing very specific research that deliberately puts users in uncomfortable situations, you want to minimize, if not altogether eliminate, anything that compromises the comfort or safety of the participant. Part of your job as a moderator is helping the participant feel comfortable participating and providing his feedback, and ensuring that he remains comfortable—physically and psychologically—during the session.

PROTECTING PARTICIPANT RIGHTS

Most professional organizations whose members interact with nonmembers in a professional capacity, including the User Experience Professionals Association (UXPA), have a code of conduct or set of ethical guidelines to govern their members’ behavior. These guidelines are in place to ensure the ethical behavior of the professional—that is, you as the moderator—and protect the rights of the nonmember (participant).

As a user researcher, you need to keep your ethical obligations to the participant front and center throughout the session. Remember, when the participant comes to your session, he may have little to no information about what he’ll be doing and why he’ll be doing it. It’s your job to provide him with this information before he begins, and obtain his consent to participate.

This informed consent lets the participant know what he’ll be doing as part of the study, what he’ll receive for participating (compensation), and his rights—for example, to take a break or leave at any time. Even though you should have him sign a consent form before the session begins, review the key points verbally as part of your briefing at the beginning of a session.
We recommend reviewing the UXPA’s code of conduct, which outlines how to behave ethically as a user researcher. As a quick cheat sheet, here are the key things to remember when interacting with your participants:

  • Do no harm, and if possible, provide benefits. As discussed in Chapter 1, there is always some level of stress associated with participating in user research. However, your responsibility as a moderator is to keep that stress—whether physical, mental, or emotional—from reaching an unreasonable level.
  • Act with integrity. Be respectful toward the participant, no matter who he is or what he does.
  • Respect the participant’s privacy, confidentiality, and anonymity. Make sure he provides his informed consent, and that he knows the steps you’ll take to avoid disclosing his personal information.

Ensuring the participant’s physical comfort is usually fairly easy to accomplish. Let him know that he is welcome to take a break at any time and, if possible, offer him a beverage. If he seems uncomfortable in the research space—for example, because the chairs are at an awkward height or size for him or the air conditioning is running high—make whatever adjustments you can to keep the environment as welcoming and stress-free as possible.

“Pay attention to context when evaluating the participant’s comfort level. He may be sweating because he is nervous, or because the thermostat is set too high. But if he is sweating and calling himself stupid, he is probably uncomfortable.”

The participant’s psychological comfort can be more challenging to maintain, especially when he is asked to interact with a product that has major usability issues or he is being interviewed about a sensitive or private situation. However, keep in mind that if a participant is struggling or growing frustrated with a usability study task, his comfort is not necessarily being threatened. In fact, that struggle often fulfills a common goal of user research, which is to identify problem areas. The participant’s struggle serves the purpose of making the product better for future users.

If, however, the participant berates himself for being stupid or unable to figure something out, or exhibits more than one of the signs of an uncomfortable participant (Table 2.4), you need to alleviate his discomfort. Remember that a participant who leaves a session feeling disgruntled or unhappy is an unhappy customer or potential customer. Given how easy it is to share negative reviews and experiences via social media, you want to avoid any potential damage to the credibility of your research or your organization.

If the participant seems uncomfortable, be careful not to assign any blame to him. Remember that he is likely in an unfamiliar environment (to him), or may be stressed to show a stranger (you) around his comfortable environment. He may be under a lot of stress unrelated to the session, such as childcare issues or tight work deadlines. Even if there is nothing external causing his discomfort, the research session itself can create some stress no matter what you do to avoid it.

Table 2.4—Signs a Participant May Be Uncomfortable

  • He increases his physical distance from what’s being evaluated—for example, chair pushed away from the table or keeping a physical product at arm’s length.
  • He crosses his arms.
  • He blames himself or apologizes for being “stupid” or inexperienced.
  • He has difficulty speaking clearly.
  • He exhibits signs of emotional distress, such as tremors, crying, or excessive sweating.
  • He constantly glances around the room or at the cameras.
  • He asks to end the session early.
  • He pays excessive attention to distractions—for example, his email is open in the background and he keeps checking it.

Watch Video 1 to see an example of a moderator interacting with a self-blaming participant during a usability study. The participant is very nervous about the session and is looking for affirmation.

Visit our Web site.

“Even if you tell the participant upfront that the test refers to testing the product, he may not internalize this. He may stay on edge if you continue to refer to the study as a test.”

To help the participant feel more comfortable:

  • Reassure him that he is not being tested, and that anything that frustrates him is valuable information for the product team. Remind him that his feedback will make the product easier to use for future users. For a contextual inquiry or interview, you can reassure him that his feedback is confidential and will ultimately be used to make his own life/work better.
  • If you’re running a usability study, avoid using the term test when referring to your research. A test is something that can be failed, and may place additional pressure on the participant. He may not understand that you’re testing the product and not him. Instead, use study, session, or evaluation of <product>. Likewise, you may consider calling each usability task a scenario instead of task.
  • If the participant crosses his arms or moves away from you—or the product being evaluated—ask him to tell you what he is thinking. This question gives him permission to express the reasons for his discomfort, and may reveal if he is uncomfortable because of something you asked him to do.
  • Ask the participant if he would like to take a short break, or go ahead and suggest a break.

If the participant reaches a point where he asks to leave or end the session early, you’re obligated to wrap things up. Be sure to give him his agreed-upon compensation and thank him for his time and help. You can check on him afterwards—using whatever means you originally scheduled him for the session; if he was scheduled by a recruiter, ask the recruiter to follow up. If appropriate—for example, if he was distraught due to external circumstances—you can even offer to reschedule for a more convenient time.

SETTING CONTEXT FOR YOUR BEHAVIOR

The first few times a usability study participant hears you respond to his questions with questions (“What were you expecting?”), or when you do not respond at all, he may become unnerved (“I don’t know, that is why I’m asking you!”), or even sarcastic (“You’re not going to tell me, are you?”). Your lack of response may lead him to feel that he is being tested, as much as you say otherwise.

You can set the context for your behavior at the beginning of the session, and reinforce it along the way. In your presession instructions, tell the participant something like, “If you ask me questions, I may not really answer you or I may be vague. If I’m doing that, I’m not being unfriendly; I’m just trying to stay neutral….” Another helpful thing to say the first time he asks you a question is, “So I’ll actually turn that question around on you; what would you expect to happen there?” By doing this, the participant can start anticipating your moderating approach and will feel less like he is being evaded or tricked. Also remember that your paralanguage—how you say what you say, like your intonation, loudness, or tempo—can have an effect on the message communicated. There is a big difference between responding to a question with, “What do you think?,” “What do you think?,” and “What do you think?” The first may come across as inquisitive, whereas the second and third might come off as rude or patronizing.

If a participant keeps asking questions over the course of a session, we sometimes resort to subdued humor to make light of the situation. For example, we may kindly say, “I’m going to plead the fifth here—like I said, I’m just here to observe. Tell me what you’d expect it to do?”—works if you’re in the United States—or “I’m going to turn that around on you again! What do you think?” Again, if your delivery and paralanguage is kind and not patronizing, usually the participant will laugh along, stop expecting answers from you, and resume voicing expectations on his own.

2.5 Use careful language and tone to probe on the situation and begin to resolve it

“One of the most important principles in moderating is to let the participant do most of the talking, but when you do speak—whether it’s to provide direction, probe for more detail, or give neutral reassurance —you need to choose your words, and your tone, carefully.”

Once you’ve established that you’re not causing or magnifying an issue, you’ll probably need to talk with the participant to try to understand more about, and hopefully resolve, the situation. One of the most important principles in moderating is to let the participant do most of the talking, but when you do speak—whether it’s to provide direction, probe for more detail, or give neutral reassurance—you need to choose your words, and your tone, carefully.

To choose the appropriate language, we suggest reviewing and memorizing some prefabricated responses that represent moderating best practices. Prefabricated responses are valuable for interacting with participants while maintaining neutrality and consistency. By memorizing at least some of these phrases and reviewing a list before your study, you save the cognitive effort of improvising appropriate responses on the spot. As we start reviewing specific situations in Part 2, you’ll be amazed at how flexible these short phrases can be.

The phrases we use most often include:

  • “What would you expect?”
  • “What are you trying to do right now?”
  • “What are you thinking right now?”
  • “How did this compare to your expectations?”
  • “What are your thoughts on how that worked?”
  • “That link isn’t working for today’s session. What would you expect to happen if you clicked it and it was working?”
  • “Thank you for that feedback, but for the sake of time, please move on to….”

(For more examples of these phrases, see Appendix A.)

“Saying something is ‘helpful to know’ is usually received as more of a token of gratitude for feedback, whereas ‘good’ or ‘great’ can be perceived as qualifying the participant’s feedback.”

Since you need to be careful about providing positive feedback to participants, we’ve found the following phrases to be relatively neutral. When used judiciously, the phrases are capable of acknowledging to the participant that you’ve heard his feedback and are giving him a small amount of reassurance that the feedback was helpful:

  • “Okay.”
  • “Okay, thank you,” or “Okay, thanks for that.”
  • “That is good to know,” or “That is helpful for us to know.”

Careful use of these phrases will address the majority of typical issues you see during your research. Just as important as what to say is what not to say, which we give examples of in the sidebar “What Not to Say.” Of course, you’ll encounter situations where these neutral phrases may not cut it, or if a participant does’t act as expected when given one of those phrases. In these cases, keep in mind these three guidelines:

  • Be as unbiased and nonjudgmental in your response as possible. No matter how inflammatory or provoking the situation is, or how much you agree—or disagree—with what the participant is saying, it is your responsibility to remain neutral. Remember that it’s not just what you say, but also how you say it, both in intonation and body language. For example, if the participant is describing a personal or cultural belief that you find repugnant or inappropriate in response to your interview question, try to keep your face still and your voice at an even neutral tone as you ask your next question. (We discuss this kind of situation where a participant makes inappropriate comments in section 13.1.)
  • Build on what the participant says rather than negating it. As mentioned in Chapter 1, one of the fundamental principles of improv is to never disagree with something that has been said, but instead to say “Yes, and…” This principle means that you’re always building on what someone else is doing rather than negating it. In a research session, your “Yes, and…” is less about agreeing with what the participant has said and more about acknowledging that he is been heard. For example, if the participant asks if he has performed a task correctly, respond with a prefabricated phrase instead of ignoring the questions. For example, “So I’ll actually turn that question around on you: Do you feel that you’ve found the information that you’re looking for?”
  • Be as graceful as possible when redirecting or reacting. When the participant wanders off track or no longer provides useful feedback, your paralanguage will set the stage for his reaction. You don’t want to imply that he has done something wrong. Here’s one of our favorite examples of a moderator-participant exchange that we’ve seen, showing how your word choices and intonation can have a negative effect:

Moderator: How would you do <task>?

Participant: Oh, I would do <this>.

Moderator: (pause, intonation rising) Oh. You would?

“WHAT NOT TO SAY
There are some phrases that you need to be careful with, as their connotations may have unexpected consequences on the participant. Table 2.5 lists these phrases, and why they usually don’t fare well.”

The rising intonation implies a questioning of the participant, and may cause him to feel judged or that he should rethink his answer. Instead, a graceful reaction would be to say “okay” with a neutral tone and move on, or ask him if there are other things he would try if the first attempt didn’t work. If he is interacting with a prototype, you could also let him attempt the task, ask how it compares to what he expected, and then ask if there are any other ways he might try to do the same thing.

Table 2.5—Examples of What You Should Not Say While Moderating

What Not to Say Why

“Let me give you a hint.”

Reinforces that the participant needs help.

“Why would you want to go there?” or “What made you do it that way?”

Implies that the place the participant wants to go, or did go, is not correct.

“Talk to me about… ”

Depending on how it’s said, or how much it’s overused, can sound commanding or condescending.

“No, don’t <do/click> that.”

Reinforces that the participant is doing something he shouldn’t.

“We/they did that because <special case could happen or special constraint limited us>.”

Implies that you were part of the design team, and/or are defending the design choices.

“That is great feedback.” or“That is good feedback.”

Implies that other feedback is not great.

2.6 Regain control to bring the session back on track

“As the moderator, you’re always responsible for maintaining control of the session. This means accepting responsibility for changing the tone or providing whatever support is necessary to bring the session back on track.”

Hopefully, what you’ve done so far has jumpstarted the session. However, if the participant feels out of control or continues in a direction that is not useful to your research goal, it’s your job to steer him back. As the moderator, you’re always responsible for maintaining control of the session. This means accepting responsibility for changing the tone or providing whatever support is necessary to bring the session back on track.

Remember that you’re not the participant’s buddy. You didn’t set up this time with him for a gripe or therapy session. It’s your job to keep him from treating it that way. At the same time, some of the art of moderating includes redirecting a session and treating the participant’s feedback with grace and respect.

So how do you reestablish control of the session? For most one-on-one methods, a specific restatement or refocusing of a task or question is enough to get the participant back on track:

  • “Thank you for going into that level of detail. For the sake of time though, I’d like you to return to <attempting/answering task/question>….”
  • “Thank you for explaining that, it’s helpful for us to hear. We’re a bit pressed for time so I’d like to direct your attention to….”

You can also bring the participant to a different task or question if you think he may be getting stuck or hung up on something in particular. (The pattern for redirecting the participant is more fully explained in section 3.3.)

If your first attempts to regain control don’t work, you may need to use stronger language to rein the participant back in:

  • “I apologize for interrupting, but it’s really important that I get your feedback on this <task/topic>. If you don’t mind, let’s return to that, and if there is time at the end of the session, we can return to this discussion in more detail.”
“Check that your body language projects an appropriate authority level. Maintain eye contact with the participant and angle your body toward him—this will help draw his attention and increase the odds that he’ll pay attention to you.”

Also, check that your body language projects an appropriate authority level. Maintain eye contact with the participant and angle your body toward him—this will help draw his attention and increase the odds that he’ll pay attention to you. However, never touch the participant, even as a way to get his attention or give him comfort. Remember that many people do not like being touched by strangers and the inherent power imbalance within a moderating session can make any touch completely inappropriate. (We talk more about how to use your body language in section 3.5.)

If the session is veering out of control through no fault of the participant—for example, technical problems—avoid sharing your frustration with him and do your best to keep that emotion out of your voice. The participant may feel like you’re directing that irritation at him.

Another reason you may need to regain control is when the participant does not meet the desired criteria and you must adapt accordingly. One approach to this situation is to transition the session into a different type of research. This works especially well if the participant is an important customer, or part of a user group, who you want to feel validated and listened to. For example, a usability study with a customer who does’t match your target user profile may be transitioned into an interview that will provide more information about his profile. Be sure to discuss these options with your stakeholders so they know what’s happening and can provide input on other types of feedback they might be interested in receiving. If, however, the participant will be unable to provide any kind of useful input—for example, if his role is entirely unrelated to your target—you may need to politely end the session. This technique is also referred to as shifting the focus, which we discuss more in section 3.8.

No matter what situations you encounter, the steps discussed in this section should help you figure out the best approach to address and resolve them. When in doubt, think back to what you would want done if your positions were reversed—how would you want your moderator to handle the situation? Chapter 3 details some of the specific interaction patterns that you can apply to help with this resolution.

SURVIVAL STORY: “I KNOW WHAT SHE NEEDS!”

Carolyn Snyder

I was running a paper prototype usability study at the client site. It was codiscovery. User #1 had already shown up. User #2 arrived with her angelic, sleeping two-month-old baby in tow. She was surprised when I reminded her that the session would take two hours. She was under the (sleep-deprived) impression it would just take a few minutes.

What I should have done was explained that we couldn’t accommodate infants, apologized for the misunderstanding, paid her, and sent her home. Instead, I asked the client if they were okay with a baby in the room. They were surprised, but didn’t object. (For the record, most of them were mothers themselves, and User #1 was female.)

Halfway through the session, the baby began crying. “I know what she needs!” the new mom proclaimed. Trying to be delicate, I asked, “Would you like privacy?” (For those keeping score at home, this was my second mistake: I should have taken her to another room.) “Nope, I’m good,” replied the mom, and put her baby to her breast right there in front of me, the other participant, and a handful of in-room observers. For the record, I am a proponent of public breastfeeding, but along with many other behaviors, it falls under the category of “Things I Would Never Do at the Office.”

I spent the rest of the session agonizing over how the client would respond. They chose not to make an issue of it—I have wonderful clients—but even now, nearly 20 years later, I still think about all the ways it could have ended badly.

It did cross my mind to send her home. But in the moment, it was surprisingly hard to choose that fork in the road. Being an eager young user experience consultant, I put a higher priority on getting feedback for the client than on the comfort of the participant, client, and even myself. I lacked the confidence to make a unilateral decision and take responsibility for it. I do still make mistakes, but I’ve learned to own them.

Discount for UXmatters Readers—Buy The Moderator’s Survival Guide: Handling Common, Tricky, and Sticky Situations in User Research and other Morgan Kaufmann titles on the Morgan Kaufmann site, using the discount code PBTY14, and save 25% off the retail price.

Join the Discussion

Asterisks (*) indicate required information.