Book Review: Paper Prototyping

Content

This book contains all of the information you’d need to create and test a paper prototype, but some of that information is presented in a narrative style that makes the book more difficult to use as a reference. That said, the author’s storytelling style is engaging and easy to digest, and much of the information is chunked for easy skimming.

Most of the content in this book is great, but Part IV seems not to be wholly relevant to the topic of paper prototyping, and there are a few other sections of the book that could have been improved by some trimming.

What Is Paper Prototyping?

Here is Carolyn Snyder’s definition of paper prototyping:

“Paper prototyping is a variation of usability testing where representative users perform realistic tasks by interacting with a paper version of the interface that is manipulated by a person ‘playing computer,’ who doesn’t explain how the interface is intended to work.”

Paper prototyping provides many benefits. According to Snyder, paper prototyping

• “is fast and inexpensive”
• “provides substantive user feedback early in the development process—before you’ve invested effort in implementation
• promotes rapid iterative [design] …
• facilitates communication within the development team and between the development team and customers
• does not require any technical skills …
• encourages creativity in the product development process”
• “imposes relatively few constraints on a design”
• “gets users and other stakeholders involved early in the process”

Snyder says, “In paper prototyping, you first create a realistic set of tasks and then build the prototype to support them, which forces you to visualize the entire process of what users will be doing. … It’s really walking through the tasks that allows us to see an interface from a user’s perspective. The paper prototype simply makes it possible for this to happen earlier in the development process, when it’s less painful to make changes.”

The way people respond to unfinished designs that are hand drawn on paper also provides some advantages, according to Snyder, because paper prototypes are “less intimidating than a computer,” encourage “a more creative response from reviewers,” and discourage “nitpicky feedback, because it’s obvious that you haven’t specified the look yet.” While, as Snyder points out, “A polished-looking prototype can encourage low-level feedback about the visual aspects of the design. When something appears to be finished, minor flaws stand out and will catch the user’s attention.”

“Because paper prototypes … let you see the strengths and weaknesses of a design early on, …” says Snyder, “they facilitate the kind of quantum design leaps that lead to greatly improved usability. … Paper prototyping facilitates the process of incorporating many people’s ideas into a design, and usually that’s a good thing.”

Creating a Paper Prototype

In Chapter 4, “Making a Paper Prototype,” Snyder provides excellent, detailed information about crafting a paper prototype, including a list of the office supplies that she uses. She creates a poster-board background that underlies all of the prototype pieces and represents an operating system or application on a computer screen or other digital display. Snyder gives many practical tips for prototyping interface widgets, representing users’ choices, and simulating interaction.

Describing a technique she learned from Jared Spool called incredibly intelligent Help, Snyder says, “When users get stuck, the facilitator prompts them to ask a question about what’s confusing them. One of the product team members …[playing the role of the Help system] gives a terse answer. … If the users are still confused, they can ask another question and the Help system will provide a bit more detail…. The purpose of incredibly intelligent Help is to find the piece of information that makes the light bulb go on in the user’s head…. Write down the questions users ask and the explanations given. After the test, see whether the information users needed can be incorporated directly into the interface … or, if that is not practical, …[include it in the Help].”

Planning a Usability Study With a Paper Prototype

Throughout Part II of the book, Snyder delves deeply into the fine points of usability testing with paper prototypes, providing many practical tips. In Chapter 5, “Planning a Usability Study with a Paper Prototype,” Snyder provides an overview of her usability process, as follows:

• Hold a kickoff meeting.
• Determine the goals of and plan the usability study.
• Define user profiles and recruit characteristic users for the study.
• Create realistic tasks users will perform during usability tests.
• Create the prototype pieces users need to perform those tasks.
• Before testing begins, have the team do walkthroughs of the testing process.
• Conduct usability tests, recording the usability issues that were discovered during each test.
• Refine the prototype following each test.
• Hold a debriefing meeting, during which the team prioritizes unresolved usability issues.
• Determine solutions for those issues or log issues for later resolution.
• Communicate the results of the study to the broader organization.

Snyder states that “all usability problems have one root cause: The development team didn’t know something that turned out to be important. This ignorance might be about what users needed, how they would interpret a term, how they would behave, or any other sort of missing information or incorrect assumption. Until you start doing usability testing, you don’t know what you don’t know.”

In discussing risk management, Snyder says, “The real value of paper prototyping comes from its ability to point out problems early in the development process, while it’s still easy to avoid them. Therefore, you should design your usability tests around whatever aspects of the user experience you are most worried about…. … Paper prototyping is an excellent tool for risk management, because it helps you clarify what you do and don’t know about how well your interface will work, and it can help the team make important decisions. … At the kickoff meeting, ask everyone … what things they’re concerned about and/or would like to know more about.” The kickoff meeting provides an opportunity for discussing business risks, new features, what important design decisions remain unresolved, what features of a product have elicited the most negative feedback, and “what tasks are critically important to users.”

Snyder offers this advice about scheduling tests: “It’s important to leave time between tests to review what you’ve learned and make changes to the prototype.”

According to Snyder, good user profiles “consist of no more than a handful of factors, …distinguish between requirements and nice-to-haves, and …mention characteristics to screen out.” “Testing with five to eight users will provide enough data for you to see the main patterns, provided that you use roughly the same set of tasks …and the users are from the same profile.”

Designing Tasks for a Usability Study

Chapter 6, “Task Design,” describes “one of the most important aspects of usability testing” in great detail. This is one of the best chapters in the book. Snyder says, “Task design is often the most difficult activity in a paper prototype usability study, because it tends to expose missing or conflicting assumptions that team members have about the product and what users will do with it.” She defines a good task as one that:

• leads to a goal that is important to users
• addresses questions and issues that are important to the success of a product and business
• is neither too broad nor too specific
• “has a finite and predictable set of possible solutions
• has a clear end point that the user can recognize
• elicits action, not just opinion”

“A good task should cause the user to interact with the interface, not just look at it and tell you what they think,” says Snyder. “Often, what users say they like does not reliably indicate what they can successfully use and vice versa. … They are responding to the concept of a particular feature, when in reality it may not work the way they envision.”

Preparing and Refining a Paper Prototype

Chapter 7, “Preparing the Prototype,” is replete with tips for the iterative process of creating paper prototypes and doing walkthroughs of test tasks using them, in which the walkthroughs inform the refinement of the prototype. Here are Snyder’s steps for this process:

• List all of the screens your prototype must include to support the test tasks.
• Prepare realistic data for each task.
• Create paper prototypes of all of the screens.
• Organize the prototype pieces.
• Do one or more walkthroughs of the test tasks to
  • “Identify parts of the prototype that still need to be created.
  • Prepare for different paths the user might reasonably take (correct or otherwise).
  • See how the interface is intended to work….
  • Give the [person taking the role of the] Computer practice in manipulating all those pieces of paper.
  • Identify issues pertaining to technical feasibility.”
• Revise the paper prototype.
• Do a usability test rehearsal.

Facilitating a Usability Test

Chapter 8, “Introduction to Usability Test Facilitation,” offers pretty standard advice on facilitation. Snyder says, “As the facilitator, you should be the person the users talk to, and the easiest way to do that is by talking to them. But …rather than agreeing, disagreeing, or explaining, the facilitator should be asking questions, encouraging users to elaborate, and remaining neutral toward the interface and its designers. It’s easy to inadvertently give users a clue about what they’re supposed to do. … Before speaking, consider what effect it might have on the users’ behavior.” She also provides the following tips:

• “Encourage questions, but don’t answer them. … Write down all the questions that users ask.”
• “Use the users’ vocabulary.”
• “Use open-ended questions.”
• “Listen for nonspecific utterances.”
• “Learn when to shut up.”
• “Let the users decide when the task is done.”

Conducting a Usability Study With a Paper Prototype

In Chapter 9, “Usability Testing with a Paper Prototype,” Snyder describes how to conduct usability tests using paper prototypes. She says, “The observers are assembled in the room before the users come in, and the prototype is on the table, displaying the screen that we’ve decided to use as the starting point. … Before giving the users the first task, I explain what they’re looking at and how we’d like them to interact with the paper prototype and Computer.” Snyder offers these tips for facilitating usability tests using paper prototypes:

• “Have [users] show, not tell.”
• Tell users “it’s okay to write on the prototype.”
• “Watch for Computer mistakes.”
• “Handle valid, but unprepared-for user actions.”

“With paper prototyping, it’s okay to make changes as soon as you have evidence that there’s a problem. It’s not necessary to keep the prototype and the tasks the same for each test, because you’re not trying to gather statistics—you’re trying to identify and fix problems,” says Snyder. During a test, “it’s fairly simple to change wording, rearrange items …, add or remove a control, and so on. You’ll get immediate feedback on whether the change helps, and if so, it’s already in place for the next test.” Between tests, it’s important to allow time “to make more substantial changes….”

In subsequent chapters, Snyder describes the role of observers in a usability test, provides guidelines for their behavior, and explains how to capture data during a usability test, then analyze, prioritize, and document the data following a usability study. Again, this is pretty standard stuff.

Deciding Whether to Use a Paper Prototype

Part III covers issues regarding the usefulness of paper prototyping in specific contexts. Chapter 12, “What Paper Is (and Isn’t) Good For,” takes a highly analytical approach to determining whether paper prototyping is appropriate for a given project, by looking at the following dimensions of a prototype:

• breadth—How much of a product’s functionality the prototype must include.
• depth—The prototype’s level of detail and, thus, it’s fidelity to the actual user experience.
• look—The degree to which the prototype must accurately represent the product’s appearance.
• interaction—The way in which the prototype handles and responds to user input.

Snyder provides a long list of questions that are typically of interest to teams planning usability studies and indicates which prototype dimensions are needed to answer them. Snyder says, “Breadth can be incorporated in any kind of prototype. … Paper prototyping’s main strength is depth—the human Computer can simulate much of what the interface will do. … Paper prototypes don’t do so well in terms of interaction, because it’s hard for the Computer to accurately mimic exactly how a machine will respond.” How well a paper prototype represents a product’s look depends on whether it’s hand drawn or uses screen shots or highly faithful renderings of screens.

The kinds of depth issues that paper prototypes will likely find include issues regarding

• concepts, terminology, and labeling
• navigation, work flow, and task flow
• content
• Help
• feature requirements
• screen layout
• brand image

Paper prototypes may also find issues regarding a product’s look. However, paper prototypes are not very effective in finding the following types of interaction issues:

• displaying small changes on a screen in response to user actions
• the need to scroll down or up on Web pages
• working with long documents and lists
• keystroke or mouse errors
• determining whether controls have appropriate sizes
• ascertaining user preference for the mouse versus keyboard and difficulties in switching between them
• using menus
• acceptable response times

The remaining chapters in Part III discuss political issues that are factors in deciding whether to use paper prototyping—which include issues of validity, bias, professionalism, and resources—and project contexts that are and are not conducive to using paper prototyping. Snyder provides a useful checklist of the factors that make testing a working user interface or a paper prototype more appropriate.

Copyediting

Most of the information in this book is chunked nicely by section headings, bulleted and numbered lists, lists with run-in heads, tables, and boxed information, and pull quotes are used effectively. Unfortunately, the text contains a rather high number of copyediting errors.

Illustrations

Paper Prototyping is richly illustrated with many photos and sketches, plus some screen shots, charts, diagrams, wireframes, and even cartoons. The book also includes some useful document templates, sample forms, and scripts. Unfortunately, some of the photographs do not reproduce very well on the paper on which the publisher chose to print the book. A higher quality paper would greatly improve their appearance. I’m not a fan of the cartoons that Rene Rittiner created especially for this book, but that’s probably a matter of personal taste.

Book Design

While from the viewpoint of information design, this book is very well designed, I found the aesthetic aspect of the book design displeasing. The fonts the book designer chose for the title page, part title pages, chapter titles, and top-level section headings are busy and hard to read. The inverted white text on a black screen behind part of each of these headings does not help matters, and the irregularly sized blobs of black are distracting.

Web addresses for pages that provide additional information or sample documents are visually flagged as “Web Enhanced,” making this information easy to find. Tables and lists are well designed, but notes are not well laid out, and their design is particularly ineffective when notes occur within bulleted lists. The format used for anecdotes makes them hard to read—again, because of a poor font choice and a gray screen overlapping the text at the left. While “Of Interest” and “From the Field” boxes are effective devices, their design is not successful from an aesthetic standpoint—particularly boxes of the former type, which introduce many too many different fonts into the book design and have inconsistent and, in some cases, cramped layouts. The format of “From the Field” boxes does not work at all well across page breaks. Some scripts for dialogues with test subjects are poorly laid out and, therefore, hard to read.

Conclusion

This is a valuable book on an important topic by an expert in usability. It demonstrates that paper prototyping is an effective technique that is useful in many contexts and provides a complete reference on how to use paper prototypes in usability studies.

Check out the book’s companion Web site. It provides a rich trove of resources for paper prototypers, including PDF versions of templates, forms, checklists, handouts, and procedures from the book; links to Web sites where you can obtain the supplies you need to create paper prototypes; and an extensive list of references, including articles, papers, and books.