Handling the Competing Demands of Field Studies

Managing the Session

As the facilitator, you must ensure that a participant correctly performs the research activity. For example, during a contextual inquiry, you have to make sure the participant continues to performs the tasks, step by step, rather than just talking about the task in general and skipping the detailed steps. You must keep the participant in the mindset of leading the session—as a master demonstrating tasks to an apprentice. You must continually encourage the participant to keep thinking aloud and ensure he or she keeps talking at your novice level of understanding, instead of lapsing into jargon.

Assessing the Session

Throughout a session, you must constantly assess how well the session is going and decide whether to make adjustments. This includes determining how well your method is working and whether you’re getting the information you need. For example, you may ask yourself:

• Is the participant doing what I need her to do—thinking aloud, demonstrating all of the steps, and talking at the appropriate level?
• Is this really a task I need to see?
• Are there other tasks I need to see, and will we have time to get to those during this session?
• Is the participant drifting off track? Should I lead him in another direction?

Keeping Track of the Time

Part of managing a session is keeping track of how much time you have left relative to what else you need to see. You usually have a limited amount of time with each participant, so you should carefully allocate your time to ensure you get to observe the most important tasks. This means you have to sneak peeks at your watch and ask yourself:

• How much longer will this task take?
• Is it taking too long?
• Will I still have time to observe the other tasks?
• If we run out of time, which tasks could we skip?

When you don’t have enough time to complete everything you’d planned to cover during a session, you have to think back on all your previous research sessions to determine which tasks you’ve already seen enough times and those you haven’t yet learned enough about. This can help you to decide how to direct a participant to the tasks that are most important for you to see.

Managing Observers

When additional people are observing a session, you must manage their activities so they don’t have any negative impacts on it. You should introduce them to the participant, then make sure they don’t get in the way, interrupt, display inappropriate body language, or ask inappropriately phrased questions at the wrong time.

Capturing the Session

To ensure you can remember what happened during a session later on, you should capture the session in some way—usually through a combination of taking notes and photos and recording audio or video. However, it’s almost impossible to take detailed notes on everything you see and hear, while also handling all of those other competing demands on your attention. And, while it isn’t that difficult to take photos or set up an audio recorder or video camera, each is just one more activity you have to remember to perform.

Maintaining a Good Rapport with the Participant

In addition to all of those other demands, it’s essential that you maintain a positive rapport with a participant. You want the participant to trust you and feel comfortable with you, not like you’re studying him. You need to observe the proper social conventions, including maintaining eye contact, using the right body language, and indicating interest to keep the participant comfortable and talking to you.

Tips on Handling These Competing Demands

Whew! That really sounds like a lot to deal with—and to a beginner this can seem intimidating. But the more experience you get facilitating field studies, the easier dealing with these competing demands becomes. However, there are also ways to ease your burden. Now, I’ll give you some tips that will make facilitation easier for you. Following these tips will help you to focus on the more important things such as observing and listening.

Learn the Subject Matter First

Learn as much as you can about the domain, the users, and their tasks before you begin your research sessions. Start with general research about the subject matter, review any existing information about the users and their tasks; and interview stakeholders to get their perspectives on the subject matter, users, and tasks. Of course, you probably won’t have time to gain extensive knowledge, but giving yourself a head start will make it much easier for you to understand what you observe and hear during the research sessions.

Prioritize the Tasks You Really Need to See

Although field studies can be unpredictable—and you won’t always anticipate what you’re going to see in the field—try to define the types of tasks you particularly want to see and prioritize them. This helps you to decide which tasks you might skip if you run out of time and which are most important, so you can focus on those first. Be realistic about how much you’ll be able to fit into each session. Trying to pack too much into a single session can be overwhelming—to both you and the participant.

Plan for Multiple Participants to Repeat the Same Tasks

In user research, you should always observe multiple participants from the same user group performing the same tasks. You should never base your findings on what just one or two people do. You must identify common patterns, problems, and themes that occur across multiple participants performing the same tasks. This also lets you see how people in similar roles do things differently.

Plus, observing different participants’ repeating the same tasks helps you to overcome some of the difficulties of facilitating field studies. Seeing the same tasks over and over again is extremely helpful when the subject matter is unfamiliar to you or the tasks are complex or fast paced. The first few times you observe a task, you might not understand everything you see and hear. You might not even realize what you don’t yet understand—or even what questions to ask—until you’ve seen the task a few times. Through repetition, everything becomes clearer. So, during the first few sessions, you don’t have to worry about whether you immediately understand each step, and you don’t have to interrupt the participants with too many questions. You can relax a bit when you know you’ll be seeing the same task many more times.

Another benefit of having many participants is that it frees you from trying to fit all of the tasks into each session. You’ll have more flexibility in deciding when to skip tasks and instead devote time to other tasks when you know you’ll have plenty of opportunities to observe those tasks with other participants.

Err on the Side of Planning Longer Sessions

It’s much better to schedule longer sessions, then end early, than it is to rush through tasks, skip tasks, or avoid asking questions, then run out of time anyway. To ensure sessions don’t feel rushed, try to get a sense of how long each task should take, decide how many tasks you want to observe, and provide extra time.

If you can’t schedule longer sessions, another option is to include more participants in a field study, then divide the tasks between them. For example, you could see tasks A, B, and C with participants 1 through 5, then see tasks D and E with participants 6 through 10.

Wear a Watch

Staying aware of time and being able to check the current time unobtrusively is critical. Unless there’s a clock in your field of vision, there’s no subtler way to check the time than glancing at your watch as you pretend to look at your notepad. You can’t subtly check the time on your smartphone. Avoid obviously checking the time or the participant may think you’re bored or distracted, assume that they’re taking too long or talking too much, or interpret your action in any of a variety of other negative ways.

Prepare Observers

Before going out into the field, set some rules for observers. Make sure they know you’ll be leading the sessions, and they’ll mostly silently observe them. Give them an appropriate time to ask questions—perhaps at the end of each session—and educate them about how to ask unbiased, open-ended questions.

Delegate Tasks to Observers

If a second or third person is attending the sessions, ease your burden by delegating some tasks to them. If you trust their note-taking ability, you can designate them as note-takers. Or perhaps you’ll want them to focus their note-taking on particular aspects such as the participants’ environment, documentation, or tools. Putting observers in charge of taking photos, recording audio, and recording video is a good way to reduce your burdens.

Take Photos at the End of the Session

Taking photos is a great way of capturing a sense of a participant’s environment, tools, technology, and documents. Taking photos is much more effective than trying to describe these things in notes, and it relieves you of the burden of trying to note these things during a session. It’s best to take photos at the end of a session, to avoid disrupting the participant’s tasks or making participants feel uncomfortable.

Record the Sessions in Video or Audio

It’s nearly impossible to facilitate a research session and capture everything you see and hear in written notes. While having a second note-taker helps, it’s also useful to have recordings. Video is ideal because you can both see and hear what happened, but recording audio is a good, low-key alternative when you’re concerned recording video might make people feel uncomfortable or it’s not allowed.

Take Only High-Level Notes During the Sessions

Regardless of whether you have a second person taking notes, as the facilitator, you should take only occasional, high-level notes during a session. These could be about high-level themes, insights, or questions or tasks you want to ask about later. Instead, rely on your secondary note-taker or your recordings to capture the details.

Later, Review and Take Notes from Your Recordings

Yes, reviewing and taking notes from your recordings can be tedious and time consuming, but do this whenever you have time. If possible, build extra time into the project plan for reviewing recordings. Perhaps you need not review each recording in its entirety, but can instead focus just on key parts of the tasks you need to review in detail.

Some people think I’m crazy to advocate this, but listening and taking notes from your recordings offers some great advantages:

• During research sessions, you can focus entirely on facilitating a session, instead of taking notes. This allows you to devote more of your mental resources to observing, listening, understanding, and asking the right questions.
• You get to see every session twice—once live, then again in the recording—enabling you to understand what you’re seeing more deeply.
• When you watch a recording, you can focus all of your attention on observing, listening, and taking notes. The need to ask questions and maintain social conventions with the participant don’t divide your attention, as they do during a session.
• You can pause a recording or go back to catch up on taking more detailed notes, as necessary. This is especially helpful when you’re observing unfamiliar, complicated, or fast-paced tasks.
• Reviewing your recordings during a field study helps you assess how well your research is going. You can reflect on what you’ve learned from each session and what else you need to learn from upcoming sessions, then adjust future sessions, as necessary.
• Reviewing your recordings helps you assess your performance as a facilitator. You can see what worked well and what mistakes you made.

Periodically Assess the Progress of Your Research

Between sessions or at the end of each day, take some time to assess what tasks you have and have not seen, what information you’ve learned, and what questions are still unanswered. This will help you decide what tasks and questions to focus on during upcoming sessions. Reviewing and taking notes from your recordings is a good way to get this perspective and make adjustments as you conduct your research.

Embrace Unpredictability

Expecting the unexpected is a good mindset to take into your field studies. Regardless of your level of preparation, you’ll never really know what you’ll experience until you’re in each session. But, when you follow the tips I’ve outlined in this column, it will be much easier to be relaxed and go with the flow when conducting field studies.

Learn and Improve from Experience

Lastly, the best way to handle these competing demands is to learn and improve from experience. The more field studies and sessions you experience, the better you’ll become at handling the competing demands of user research. Sure, you’ll make some mistakes and sometimes encounter difficulties, but you’ll learn how to adapt to them, and your confidence will grow.