Part 1 of our three-part series on continuous customer feedback programs covered how to set up a program like ours at IBM to obtain the qualitative data that your product team needs. Now, in Part 2, we’ll describe how to conduct customer feedback sessions, including
leading discussions to find the answers that you need
involving participants
involving product team members
Leading Discussions and Finding Answers
During a customer feedback session, it’s tempting to make a presentation, turning your session into a Show and Tell session. But, if you just show participants an exciting new feature, ask them whether they like it or would find it useful, then move on to the next feature, what would you learn? Not much.
Validate, don’t just review. The main purpose of your customer feedback sessions should be to confirm whether your designs would satisfy real user goals and needs and identify what you should work on next.
Champion Advertisement
Continue Reading…
When providing a design solution for a client, ask them to describe their own process or workflow for a scenario before revealing your design ideas. Get participants into the mindset of thinking about their own experiences to ease them into the conversation. Not only does this approach get your participants comfortable with talking, it also helps you avoid any bias from their having seen or heard your ideas. Be careful, however, to keep the discussion focused on the scenarios that comprise your agenda.
The earlier during a session that you solicit user input, the better. So, don't worry about the fidelity of your visual aids. Low-fidelity drawings or computer-generated wireframes may actually prompt more feedback and discussion than high-fidelity screen mockups would. Presenting high-fidelity mockups can make your audience believe that the design for a feature is near completion and, hence, that you don’t really need their input. Plus, high-fidelity screen mockups typically take more time to compose than incomplete wireframes do.
If a design fails to meet users’ needs, you’ll be happy that you didn’t spend more time perfecting it. There is far more value in testing an early idea than in having an implemented feature at this stage. Often, by the time a feature gets implemented, it’s too late to integrate user feedback. Instead, show participants just enough to foster a vibrant discussion.
Sometimes you’ll need information before you can put together a new design—in this case, no visuals at all are necessary. During these sessions, you’ll encourage participants to tell their stories and explain their process to you. The less there is to look at, the more engaging participants will find the questions that you’re asking them, as well as what other participants are saying.
As users come up with ideas, rather than automatically documenting their ideas as future enhancements, challenge their ideas. Ask why until you determine the root need behind a feature request. Users are always happy to tell you what they want, but it’s your job as a UX researcher to determine what they really need. If you commit to making an enhancement without understanding the need behind a request, you’ll be risking feature creep. Instead, set the expectation that group consensus will drive the enhancements that come out of the customer feedback sessions.
Ideally, your participants should do most of the talking during the sessions. Remember: these sessions are not for presentations. They’re for investigating user needs. However, when it is your turn to speak, your choice of words is critical. Learn to ask open-ended, non-leading questions, and give participants time to think and respond. For example, instead of asking whether a user would use a feature, ask him how he would use the feature. Instead of asking a user whether a new feature improves the user experience, ask her which of two features would provide more value to her experience of your product and why. Instead of describing a wireframe yourself, first ask a participant to explain how he thinks the design would work.
Your tone should be friendly and approachable, but professional. Keep your responses neutral and non-committal, so you don’t guide participants or give the impression that you’ll deliver what they’ve requested.
Involving Your Participants
Make sure that your topics of discussion reflect both what you want to investigate and what your users would find interesting. Find a way to take turns with your participants in determining your sessions’ agendas. For example, twice a year, let your participants craft the agenda for a session. Participant-driven sessions need not be design reviews. Instead, they might involve a guest speaker’s providing deep technical explanations or high-level overviews relating to your product. Your customer feedback program doesn’t always have to focus on design.
As the session facilitator, be aware that a big challenge for all facilitators is pausing during their presentation of a design often enough to allow participants to share their thoughts. Not all participants are comfortable with interrupting, so stop your presentation frequently to ask for questions or comments. Not giving participants enough time to share their thoughts is much worse than not getting through your material. Allowing participants to provide their feedback throughout a session not only increases the amount of data that you’ll have to work with afterward, but it also helps immensely in making participants feel respected and engaged. These feelings help to ensure that your participants return session after session.
Not only is it sometimes a struggle for participants to interrupt with a question or a thought, some participants have a hard time speaking in a group setting. So, if you haven’t yet heard from a participant, address that person by name, either asking a specific question or saying something like, “Joe, we haven’t heard from you yet. What’s your opinion or experience?”
On the other hand, some participants have trouble allowing others to speak and interject their thoughts too often. To prevent this, it’s helpful to present your agenda at the start of a session, with approximate times for each topic of discussion. When you have an agenda to which you can refer, this makes it easier to suggest taking a topic offline that is derailing your discussion—citing the need to progress through your agenda. However, if you use this technique, be sure to follow up with any participants who may have felt that they didn’t get to provide their complete feedback after the session is over.
If you end up with a very vocal participant who is having a detrimental impact on your session, consider setting up annual enrollment for your customer feedback program, requiring users to apply to join your program. This might give you the opportunity to decline requests from participants who have damaged the effectiveness of prior sessions.
Along with showing your agenda, consider stating the purpose of your customer feedback program at the start of each session. Stress that the purpose of these sessions is not for venting about topics of the participants’ choice. Rather, you’ll be discussing specific, new designs to ensure that you can provide the best experience moving forward.
Not only will your participants want their voice to be heard, they’ll also want to feel that they are cutting-edge insiders. From time to time, remind them that the designs they’re seeing are new and exciting ideas that are not yet available to the rest of the world.
Both while preparing for your sessions and during your sessions, it’s always helpful to ask yourself, “What’s in this for our users?” Is it clear that you’re listening to them? Are your topics of discussion interesting to them? Have you reminded your participants lately that their feedback is helping you to build the product that they really need and want? Are you giving them early visibility into new ideas and designs? If your users feel that they’re getting value out of your customer feedback program, they’ll keep coming back.
Involving Your Product Team
Involving your product team in customer feedback sessions is a great way for them to learn about real user needs. Hearing opinions directly from real users helps to remind team members for whom they’re building the product.
Developers can do an excellent job of presenting your team’s designs if you provide some coaching. Before each session, share tips on asking open-ended questions, letting participants speak, and refraining from defending a design solution. It’s helpful to provide developers with a presentation template to prevent their presentations from diving into technical details or lapsing into Show and Tell. The presentation template should guide a presenter in asking the following questions about each feature:
What is the feature’s purpose?
What personas would find it useful?
In what situations would people use it?
Ask your developers to complete their presentations in time for you to review them before they present during a session. Work with them to develop a story around their demo that puts a feature into context. Be sure that developers know how much time they will have to present.
Occasionally, including a development perspective in your sessions can be immensely valuable to your participants. Not only do they get some time from the person or team developing the feature, they also have an opportunity to ask questions about its implementation and intended use. This can be an enlightening discussion for both the developers and the participants.
To give participants a long-term view of what’s to come, product managers can offer their product strategy and a roadmap including major milestones. Presenting a roadmap can help you to steer the product in a direction that would bring the most value to the market and to users. It’s great when the product manager can get that feedback first hand.
Since your product manager might not have time to attend every session, it’s wise to get his or her approval of your discussion topics to ensure that you’re exposing the right ideas to participants, at the right time.
There is value in every product team member’s attending your customer feedback sessions, but be careful to ensure that, in a given session, your team members don’t outnumber your participants. You want participants to feel that they’re in control and can speak freely. Participants don’t want to feel like they’re under a microscope. Consider inviting to a session only those team members who are presenting or leading a particular part of the discussion. Record all of your sessions, then either distribute the recordings to the rest of team or listen to the recordings together as a group. They’ll help foster useful discussions.
Coming Next…
In Part 3 of this series on continuous customer feedback programs, we’ll discuss post-session activities, including processing and organizing the data from the sessions, communicating customer feedback to your team, and establishing priorities for development. We’ll also describe the benefits of continuous customer feedback programs and how to sustain them.
After graduating from the University of Waterloo’s co-op Computer Science program in 2000, Marnie spent her first 8 years working in the software industry splitting her time between programming, interaction design, and usability at small companies that valued her ability to move between skills. Then in 2008, Marnie became a full-time UX Designer at IBM. With her passion for designing with real users, she has led cross-disciplinary product teams to improve the quality of their deliverables by leveraging users’ feedback. Now, as a Design Manager, Marnie ensures that the right people are on the right projects. Her team creates design solutions using the best methods possible. Marnie is currently creating a design research program with her team at IBM Business Analytics. Read More
At IBM, April manages usability activities and deliverables for an enterprise quality-management product. Her day-to-day work includes conducting usability research, creating user-centered design deliverables, engaging in extensive cross-functional collaboration, and transforming client feedback into actionable UX design and development work. April’s academic background includes a BS in Information Design and Corporate Communication and an MS in Human Factors in Information Design, both from Bentley University near Boston. Read More
