Analyzing Data from Field Studies | Wiki Design

Daniel recommends that you read some of his writings on this topic:

• “Finding Gold in Your User Research Results”
• “The Value of Asking ‘Why?’”
• “The Power of Field Studies”

Daniel and Pabini also recommend Steve Baty’s “Deconstructing Analysis Techniques.”

“We’ve published many excellent articles on this and related topics on UXmatters,” says Pabini, “including the following—in addition to those Daniel has already mentioned:

• About data analysis and synthesis:
  
  • ‘Analysis, Plus Synthesis: Turning Data into Insights,’ by Lindsay Ellerby
  • ‘Patterns in UX Research,’ by Steve Baty
  • ‘Sample Size Oddities,’ by Steve Baty
• About user modeling and task analysis:
  
  • ‘Audience Segmentation Models,’ by Steve Baty
  • ‘Hierarchical Task Analysis,’ by Peter Hornsby
  • ‘User Research for Personas and Other Audience Models,’ by Steve Baty
  • ‘What’s My Persona? Developing a Deep and Dimensioned Character,’ by Traci Lepore

“Plus, in my recent column, ‘Design Is a Process, Not a Methodology,’ the section ‘Discovery Phase’ delves into these matters in some depth.”

Managing and Analyzing Your Data

“Is the question really how to manage the mass of photographic, video, hand-written notes, and audio recordings?” queries Whitney. “Because, otherwise, the problem is the same as analyzing any user research data—just, sometimes, more of it.”

To manage and assess your observations from field studies, Daniel suggests, “One of the best methods is to take your observations and put them onto Post-it notes. Find a large wall and put them on the wall to look for patterns.”

“When you have a large amount of unwieldy data, affinity diagramming lets you use the natural relationships between your various observations from field studies or other types of user research to organize them into meaningful and actionable groupings,” advises Pabini. “Once you’ve completed your user research, follow this affinity-diagramming process:

1. Record each of your observations on a Post-it note or note card.
2. Stick all of your notes up on a whiteboard or wall or lay them out on a large table.
3. Working collaboratively with your UX team or product team, consider which observations are related to one another and gather all of your observations into meaningful clusters. Make sure you get consensus on these clusters.
4. If particular clusters are too large to be manageable, break them into subgroups.
5. Discuss the theme of each cluster, then write a descriptive name for it on a large Post-it note and place it above the cluster.”

“There are a lot of methods for analyzing the data from field studies and many books about it,” responds Jim. “You need to find the method that works best for you. A great book that provides an overview of many different methods is User and Task Analysis for Interface Design by JoAnn Hackos and Ginny Redish. It’s a little old now, from 1998, but I haven’t found any other book since that provides such detail about a variety of different methods.”

“You’ll find good coverage of the analysis of card-sort data in Donna Spencer’s book Card Sorting and of interview data in Indi Young’s book Mental Models,”recommends Steve. “Each of these books deals with a specific type of data toward a specific purpose.”

Whitney recommends the following resources:

“Books:

• The techniques in Indi Young’s Mental Models would work for analyzing concepts.
• Mike Kuniavsky’s book Observing the User Experience has good descriptions of several user research methods and how to use the data.
• Ginny Redish and Joanne Hackos’s book User and Task Analysis for Interface Design is still a good reference on conducting and analyzing field studies.
• Any good book on analyzing ethnographic data would also offer good insights—for example, David Silverman’s Interpreting Qualitative Data: Methods for Analysing Talk, Text and Interaction, but it focuses primarily on textual data, as its title suggests.

Training:

• Susan Dray and David Siegel teach excellent workshops on analyzing field data at both CHI and UPA.
• Kate Gomoll has taught a great workshop on both gathering and presenting field-study data.”

Wiki Design

Q: Our software development team is utilizing a wiki to support the development of our large project. However, we are struggling to find our information in it. Do you have a favorite wiki-building tool? Do you have any general advice for organizing and finding the content in a wiki?—from a UXmatters reader

“Wiki’s allow anyone on a team to easily edit their structure and content,” observes Peter. “From a UX perspective, this is both a strength and a weakness: your wiki can quickly become populated with useful user-generated content, but you also run the risk of your content becoming increasingly inaccessible over time.

“I think you have a great opportunity here to engage with the software development team and use UX techniques to create a structure: card sorting could be a very effective approach here. If people are generating content in an ad hoc way, with no overarching direction—as is very likely—it may be useful to commit to having periodic reviews of the structure where representatives from each team—who should also be users—sit down and agree on how they should restructure the content. The risk here is of the loudest voice in the room dominating, but this is a normal challenge for UX! If there is some existing structure you can use—such as your organization’s processes for software or project development—these can provide a useful framework on which to structure your information. The sooner you act, the easier your task will be!”

“I’ve been using Google Sites to communicate with product teams and provide and organize resources for them for several years,” answers Pabini. “It’s highly customizable and provides good authoring tools. Having a wiki can be helpful for any team, but it’s especially important for teams working remotely. However, since wikis often grow organically, they can quickly become unnavigable, as you’ve observed.

“I’ve found that building in good structure from the beginning helps a lot. Generally, that means someone who has information architecture (IA) skills and knows the capabilities of the wiki tool well should develop an overall framework within which the wiki can grow—and perhaps even build landing pages and stub pages it’s likely the team will need. Often, it’s a good idea to create sections of the wiki for the various disciplines on a product team—like Product Management, User Experience, Development, and Quality Assurance. Creating templates for specific types of pages also helps bring consistency to your wiki at the page level, making it easier to find information within pages and ensuring people add the proper information to pages.

“Someone within each discipline can act as a keeper of their section of the wiki, so if the wiki’s structure starts disintegrating over time, he or she can restore order. This sometimes means moving information that someone has placed on a page where it doesn’t really belong to its own page and adding it to the navigation bar, so people can find it. Sometimes it’s necessary to move pages to maintain a logical hierarchy and usable navigation. Alternatively, representatives from each discipline can work on the entire wiki’s information architecture collaboratively, as a team.”

“I’m using EditMe,” says Caroline. “Whitney Quesenbery recommended it to me. I find it quite easy to use and the support has been good.”

“It seems to be a common fallacy that using a wiki to allow a group to create and edit a collection of information guarantees that it will be structured logically and linked well,” replies Whitney. “In my experience, it takes both some initial work to agree on how to organize the pages—just as with any content organization—and a periodic tidying up. One way to do this is to designate someone—preferably someone with some IA experience—to manage the site. This might mean creating new pages for editing, creating navigation links and landing pages, or even reorganizing information within the pages.

“Most of all, it seems to take some explicit agreements about how people will add information to the site, and a shared understanding about page titles, section headings, and even how much goes on a page. It’s not magic, just housekeeping. The work doesn’t happen by itself. When I’ve seen it work well, it’s been a conscious part of the team process.” 

References

Hackos, JoAnn, and Janice Redish. User and Task Analysis for Interface Design. New York: Wiley & Sons, 1998.

Kolko, Jon. Exposing the Magic of Design: A Practitioner’s Guide to the Methods and Theory of Synthesis. Forthcoming from Oxford University Press, in late 2010.

Kuniavsky, Mike. Observing the User Experience: A Practitioner’s Guide to User Research. San Francisco: Morgan Kaufmann, 2003.

Silverman, David. Interpreting Qualitative Data: Methods for Analyzing Talk, Text and Interaction, 3rd edition. London: SAGE, 2006.

Spencer, Donna. Card Sorting: Designing Usable Categories. Brooklyn, NY: Rosenfeld Media, 2009.

Young, Indi. Mental Models: Aligning Design Strategy with Human Behavior. Brooklyn, NY: Rosenfeld Media, 2008.