Top

Continuous Customer Feedback Programs, Part 1: Getting Started

June 17, 2013

In this first part of our series of articles about customer feedback programs, we’ll describe how to take the classic focus group and turn it into a continuous customer feedback program—a program of recurring sessions that feed your product team the qualitative research it needs. We’ll draw from our own experience running such programs at IBM.

Champion Advertisement
Continue Reading…

The goal of a continuous customer feedback program is to engage real users in conversations about your product. As a UX researcher, you can conduct a customer feedback program to accomplish the following objectives:

  • Identify and verify customers’ goals, behaviors, and attitudes.
  • Explore and validate upcoming designs and other user-centric deliverables such as personas and scenarios.
  • Discuss the pain points of existing product designs.

You can use the findings from these conversations to support the design recommendations that you provide to your product development team.

Why Create a Continuous Customer Feedback Program?

Basing your designs on users’ real needs is critical to your product’s success in the marketplace. Equally important is ensuring that your product is easy to use before it’s developed—let alone released. The fact is that, if people don’t want to use your product or can’t use it, they won’t use it.

By collaborating with people who really use your products, you can explore and validate your product’s design and ensure the best user experience possible. However, having a real user as part of your sprint team is often an unrealistic luxury. One-on-one sessions are great, but are often time consuming to plan and execute. Sporadic or ad-hoc sessions can be great—if you’re disciplined enough to fit them into your plan often enough. But it’s easy to get stuck in design mode and forget to pause and collect feedback from real users or just skip getting users’ feedback altogether.

Setting Up Your Customer Feedback Program

When you ask something of your customers, you must provide something to them in return. Holding recurring customer feedback sessions with users ensures that you keep them close and convinces them that they’re being heard. Careful preparation is necessary to encourage attendance and foster open communication with users. Here are the tools, tips, and techniques we’ve found to be effective in setting up our customer feedback program.

Legalities

Before you begin, consult your legal team to understand how you can communicate with your customers and what documents need to be in place for customers to participate in your program. For example, customers may need to sign a nondisclosure agreement so you can discuss new designs or product futures that are not yet public knowledge.

Your legal team might also have a say in how you can recruit participants. For example, are you allowed to broadcast your program on Facebook?

Ask your legal team to compose any disclaimers that you need to provide or state at the start of each session with users. Often these disclaimers emphasize the importance of not basing any purchasing decisions on session materials or discussions.

What Users Should Participate?

It’s important to identify the right people to join your customer feedback program. First, you want people who closely reflect your product’s personas. In some cases, your contacts will be the people who manage such users, so encourage them to identify real users who have time to participate in your discussions. If you’re unable to get that commitment, accept the participation of managers, but recognize that they’re speaking on behalf of real users.

Not only should you consider participants’ personas, you should be sure to give participants a positive experience. It’s important to set expectations about what participants will give to and receive from the program. Make it clear that they should not use the sessions to vent about your current product and that you’ll guide discussions through a set agenda. In addition to conveying this information to participants, make sure your extended team understands that the explicit purpose of the program is not to calm irritated customers or collect information about requirements and defects.

Effective focus groups include no more than ten participants at a time. However, participants are often unable to attend every session, so aim to recruit about fifteen people. If the number creeps higher, consider breaking your program into smaller groups. You could divide participants into groups based on user type or the types of activities they perform using the product. Or you could open each group’s session by repeating the same content, then let the participants in a particular group choose the focus of their session.

Recruiting Participants

How you should get your customers involved in your customer feedback program depends on how many users your product has and how easy it is to find the right participants. Let’s look at two possible scenarios.

If your user base is small and specialized or it’s difficult to get access to the right people—for legal reasons, for example—work with your Sales and Product Management teams to help identify existing customer relationships that you can leverage. Highlighting the opportunity to influence design and how it builds customer engagement can help you sell the value of the program to these teams.

Look for customers who are participating in forums, user groups, communities of practice, and other social media for your product. But be sure to confirm with your legal department that you can recruit through these channels. Existing, past, and future customers could all qualify for your program. You could also reach out to subject-matter experts, but note that their perspective isn’t quite the same as that of most real users.

If your user base is large and easy to access, you may have the opposite problem: too much interest. In this situation, recruit volunteers who are clearly motivated to improve your product’s user interface and usability and who will speak up, while keeping the conversation productive, rather than venting about your product’s shortcomings. When selecting participants, you can advertise a limited number of spots in the program, then interview volunteers to ensure that they best represent real users. You can combine the results from these interviews with additional information such as references from account representatives or by judging the quality of their participation in other forums such as product communities. Each year, review your list of participants. If necessary, ask participants to re-enroll, then repeat the selection process.

Be the gate keeper for popular programs. If your team is inviting customers on your behalf, speak with them to communicate the importance of your being involved in participant selection, as well as keeping the number of participants manageable and recruiting the right mix of personalities.

Once you’ve attracted the best participants, maintaining their commitment will be your ongoing responsibility. Before each session, be sure to ask yourself on behalf of your users, “What’s in it for them?” Ensure that participants have a reason for continuing to attend your customer feedback sessions. Keeping your existing participants always requires less work than recruiting new participants.

On the flip side, how can you keep the feedback fresh when the same users attend your sessions for months or years on end? Making enrollment an annual activity allows you to change up the participants. If your user base is small, rather than looking for new companies, you can ask participants to allow another team member to attend the sessions.

Which Stakeholders and Team Members Should Participate?

Getting backing from your internal stakeholders can help make your customer feedback program a success. They may have customer contacts who you can recruit or suggestions for session content. Or they might drop in as guest speakers to discuss topics such as your product roadmap. Product Management can provide market insights that help you to identify hot topics and define your program personas. Development leaders can encourage feature specialists to attend sessions as researchers or speakers, on behalf of the development team.

Including your product team early in the program builds a sense of common ownership for user satisfaction. Feeding your discoveries back to your stakeholders helps to educate them about user needs and define product strategy. If they’re already fans of the program, you’ll have an easier time getting them to integrate the user feedback that you’ve collected into the product.

Frequency

The frequency of your customer feedback sessions depends a lot on where you are in the software development lifecycle. Typically, we’ve met with our customers for one hour, one time per month, during our design and implementation phases. However, once a beta is ready, we increase the frequency to once a week for the duration of the beta program.

Consider the cadence of your sprints and milestones. Ideally you’re scheduling your sessions at times when there is new material to show, as well as enough time remaining in the iteration to make changes based on participants’ feedback.

During the design and implementation phases, the focus of sessions tends to be exploration and validation. Often the product team needs time to develop theories that are based on users’ previous feedback, then create a design that they can show to customers. Because concepts tend to be more vague earlier in the design process than later on, you can often fit several topics into one session. If a topic turns into a deeper discussion, you can always book a special session one to two weeks later to finish discussing the topics. It’s best to avoid waiting a full month because you need to get your questions answered.

Sessions during the beta phase let you give back to your audience. Spending time explaining new features in depth and showing how their feedback and early design reviews have influenced the product design convinces your audience that you’ve been listening. Plus, it gives them a personal tour of the new functionality and generates excitement about the upcoming release.

Tools

Most of the time, you’ll conduct your customer feedback sessions remotely, so you’ll need Web conferencing tools. An analysis of conferencing tools is beyond the scope of this article, but in general, you’ll need a way users can make toll-free calls—either by phone or using a conferencing tool. You’ll also need video conferencing that, at a minimum, allows screen sharing in both directions, so participants can view your screen and you can view participants’ screens. Being able to see participants’ faces is an added bonus.

An online community tool that promotes collaboration outside of your sessions is also useful. There are many Web sites that let you create a private community in which users can post questions and ideas for future sessions, share examples, and pick up session agendas and materials. Check with your legal team regarding what you can share in your community.

If your community tool lets people share large files and your conferencing tool supports recording, sharing recordings of each customer feedback session can help keep participants engaged when they are unable to participate in the odd session.

What’s Next?

In Part 2 of our series on continuous customer feedback programs, we’ll describe how to conduct a great customer feedback session, providing tips on successfully leading discussions to ensure that you find the answers you need and involving participants and product team members. 

Senior UX Researcher at Nokia

Ottawa, Ontario, Canada

Marnie AndrewsAfter graduating from the University of Waterloo’s co-op Computer Science program in 2000, Marnie spent her first 8 years working in the software industry splitting her time between programming, interaction design, and usability at small companies that valued her ability to move between skills. Then in 2008, Marnie became a full-time UX Designer at IBM. With her passion for designing with real users, she has led cross-disciplinary product teams to improve the quality of their deliverables by leveraging users’ feedback. Now, as a Design Manager, Marnie ensures that the right people are on the right projects. Her team creates design solutions using the best methods possible. Marnie is currently creating a design research program with her team at IBM Business Analytics.  Read More

User Researcher at IBM

Boston, Massachusetts, USA

April L. de VriesAt IBM, April manages usability activities and deliverables for an enterprise quality-management product. Her day-to-day work includes conducting usability research, creating user-centered design deliverables, engaging in extensive cross-functional collaboration, and transforming client feedback into actionable UX design and development work. April’s academic background includes a BS in Information Design and Corporate Communication and an MS in Human Factors in Information Design, both from Bentley University near Boston.  Read More

Other Articles on User Research

New on UXmatters