Diary Studies: A Research Method Worth Trying

Broad and Targeted Diary Studies

Diary studies can be broad or targeted. If your team were working on a meditation app and taking a targeted approach, you would ask users to complete their diary entries based on their app usage alone. But, in the case of a broader study, you would also encourage them to make entries that relate to their having mindful moments when they’re not meditating using the app.

Why Choose a Diary Study?

Diary studies aim to understand what users do, think, and feel when completing a task or having an experience. They provide rich, contextual information about the customer journey, either as it is happening or shortly thereafter. These insights can unlock key information about the end-to-end user experience that, in turn, enable teams to improve the user experience, differentiate it from those of competitors, and gather data that influences the creation of or adds context to a customer-journey map (CJM). If you are trying to understand what users do and why they’re doing it, diary studies are a great option.

What Does a Diary Study Let You Learn?

By conducting a diary study, you can learn the following about users:

• Habits and usage:
  
  • When are participants using the product?
  • What prompted them to use the product? (Think of circumstances such as their receiving notifications or because of the specific time of day.)
  • How long are they using the product?
  • How often are they using the product?
  • What leads to or inhibits specific behaviors such as purchasing a product?
• Attitudes:
  
  • What motivating factors led users to perform the tasks?
  • What did users feel throughout the journey and when they completed the tasks?
  • What were users thinking during the journey and when they completed the tasks?
• How users’ attitudes or sentiments change over time:
  
  • Do customers feel loyalty to the brand or product?
  • Are they pleased with the product? Why or why not?
  • Did they like a new feature or did it turn them off?
  • Are they able to learn how to use a new feature?

When I reflect back on the printable yoga and mindfulness product line that I created, I see in hindsight that diary studies could have been instrumental in my learning how people were using the products and how actual users perceived them.

A diary study could have followed teachers and other users as they utilized the content in their classes and given us opportunities to ask questions such as the following:

• How did they use it?
• When did they use it?
• What was their sentiment about their outcomes?
• What prompted them to use the materials?
• Were there any special moments to share about students applying the learning?

Although I collected much of this data informally, my knowing how to conduct a diary study could have yielded data that would have helped me learn more about my users’ frustrations, wins, and motivations. Now, let’s look at how to conduct a diary study.

How to Conduct a Diary Study

Before you start a diary study, it’s important to be prepared. Diary studies depend on your having a well-defined list of questions whose aim is answering your research questions.

Remember, your participants are human, so respect their time. Your diary study must be compatible with the way people use your product and should occur within the context of a use case for your product. I suggest your encouraging participants to create diary entries either at regular intervals such as once daily, in the morning or evening, or after each use of the product.

• Use a variety of question types.
  
  • Open-ended questions prompt participants to expound on their thoughts, experiences, feelings, and motivations in a way that multiple-choice questions cannot.
  • Use questionnaires or prompts to ask specific questions whose aim is to help discover where there is room for improvement. Participants share the actions they took, reactions to these steps, and their assessments or thoughts about their experience.
  • Voice memos, participants’ videos, and calls with researchers can enable participants to provide their more in-depth thoughts and help researchers to see or hear participants’ emotions. Sometimes diary studies include the option to provide responses that answer more than one type of question.
• Schedule participants effectively.
  
  • Chasing participants at random intervals won’t yield the data you need. Ensure that you establish a schedule and that participants understand and consent to follow it so they can comply with the study requirements. Plan to use alerts, reminders, texts, email messages, or notifications to prompt participants to make their diary entries.

A Step-by-Step Guide to Conducting Diary Studies

To conduct a diary study, follow these steps:

1. Planning and preparation—During the planning phase, write your research questions and participant instructions, provide resource materials, and establish how you’ll collect data and prompt participants to make diary entries. At this time, consider whether to recruit participants from current or target users.
2. Participant recruitment—Recruiting and vetting participants is a critical step before your study can begin. Recruit participants based on the target audience for the specific features of your product that you want to test or evaluate. Recruiting from a high-quality candidate pool that comprises members of your niche audience is imperative because it guarantees that the feedback your team receives comes from individuals who match your research requirements. Participant recruitment and management platforms streamline this process.
3. Prestudy brief—When doing your prestudy brief, schedule a call with participants to set expectations and provide instructions. Participants have the opportunity to ask you questions and get to know you and your research team. Participants should also know when and how to submit their responses.
4. Logging period—This is the designated period for collecting diary entries from participants. Something to consider: Strike a balance between the duration of your study or the number of repeat uses of your product that it requires and the amount of time you expect participants to be willing to commit to the study. If you design a study to be too long, you may experience attrition or participants’ filling the diary with inadequately thoughtful responses. If a study is too short, you might miss out on important data about changes in participants’ attitudes over time. A good rule of thumb for encouraging participants to write diary entries over time is making sure that the incentive you offer demonstrates value for the participants’ time. Remember to monitor insights as you receive them so you know when there is a need to ask clarifying questions. Types of logs:
   
   • Text logs provide written content such as unprompted feedback about negative experiences that can help you improve the product.
   • Video or voice memos provide rich data. Participants talk you through their use of the product and explain their thoughts and feelings.
   • Photos let you see the user’s context.
5. Post-study interviews—During this interview, you ask participants questions about their overall experience with the product. This is the time for researchers to ask additional questions to clarify what participants thought, felt, and did when completing their assigned tasks.
6. Analysis of your findings—You can pitch UX design improvements and product iterations to stakeholders based on the key points of friction that participants experienced as they completed the activities that you defined in step one.

Benefits of Diary Studies

Conducting diary studies provides the following benefits:

• These longitudinal studies let you collect information over time.
• They reveal details about the user’s thoughts, feelings, desires, motivations, and painpoints in a detailed way that other user-research methods cannot.
• They provide flexible options for recording entries, including voice memos, email messages, texts, videos, and more. Technology has made dairy studies easier than in the pen-and-paper days.
• There are many ways to prompt users, including text reminders, alerts, check-in calls, questionnaires, and the participant-recruitment process. Plus, management platforms can handle automated outreach.
• Subject to participant quality, this method is very inexpensive given the richness and reliability of the data.
• Because diary studies occur in the user’s home or office, they reduce lab expenses. Plus, being at home encourages users’ more natural behaviors.
• Remote facilitation of diary studies allows many to participate concurrently, typically on their own devices.

Limitations of Diary Studies

Diary studies have the following limitations:

• It can be challenging to ensure participants are compliant with the study’s requirements. They must consistently complete their diary entries over time.
• Communication struggles such as challenges with the legibility of participants’ handwriting for handwritten diary entries, difficulty interpreting people’s strong accents when reviewing voice memos or videos, or entries from people who lack strong written communication skills. You can address these issues during participant recruitment—for example, through appropriate screening for good articulation.
• Because of this research method’s emphasis on participants’ experiences and feelings, it is not the ideal method for triangulating on flaws in designs. Ideally, you should use this method in conjunction with other user-research methods.
• You can’t always trust what participants say. They are not always honest and sometimes say what they think researchers want to hear or whatever is quicker.
• Parsing and analyzing the diary entries takes time and effort.

Participant Recruitment

The keys to successfully recruiting high-quality participants for a diary study depends on finding relevant, articulate, authentic candidates. Do the participants’ traits and experiences match important requirements? Do they have the ability to communicate at an acceptable level and show an interest in helping improve the user experience?

Participant recruitment and management can vary in complexity, depending on how specific your research needs are and how accessible your niche audience is. There are use cases that would allow UX researchers to recruit qualitative-research participants from a user pool to which they have easy access. There are other instances in which researchers would need a specific kind of user to participate—for example, banking executives who would complete a specific action using a specific feature of an app with a certain amount of regularity. In such instances, doing your recruiting using a user-research recruitment platform is cost-effective and can save your research team time, bandwidth, and energy to focus on the actual research.

Convincing Stakeholders That User Research Matters

Rather than simply restate what I wrote earlier, I scoured online research findings for meaningful metrics from thought leaders in User Experience that you can use to help you argue the case for user research as a valid investment, as follows:

• “Businesses that invest close to 10% of their total budget on UX are likely to see tremendous ROI. User research improves user experience, and it must be done right.”—Amazon AWS
• “The cost of fixing an error after development is 100x that of fixing it before development.”—“The RIO of User Experience,” by Dr. Susan Weinschenk
• “UX-centric organizations need fewer support calls, have increased customer satisfaction, reduced development waste, and lower risk of developing the wrong idea.” The UXSchool by Career Foundry

In Summary

I hope this article has helped paint a clear picture of the use cases, best practices, and operational processes for diary studies. Diary studies are a powerful research method that provides rich, reliable insights into consumer behaviors and sentiment. As UX researchers, isn’t that exactly what we want?!