Testing Your Own Designs Redux

I don’t have a great answer to that question. It’s extremely difficult for people to be objective and balanced in their criticism of their own designs—especially if you consider that their boss or client is also evaluating them on the quality of their work. As Bob Dylan said, you’ve gotta serve somebody. (I’m not a big Dylan fan, but this just seemed to fit here.)

Indeed, an anonymous commenter on my last column thought I hadn’t gone far enough in describing the pitfalls of testing our own designs, citing organizational and political factors that could affect our ability to test our own designs effectively and impartially, as well as various tricks designers could pull to stack the deck, presenting their designs in the best possible light:

“… [It is] possible that the designer may well bias the plan and the script toward things that the designer has focused more heavily on, and therefore, things that the design is more likely to perform well on.”

I think that’s a tough effect to defend against. Susan Hura recommends taking the Zen approach of a beginner’s mind when testing your own designs. I agree with her and want to add these key points: If you, as designer who is testing your own design, stay focused on the outcome—that is, a good design—instead of merely focusing on how your boss is going to perceive your initial design, and you’ve already done the work of selling management and your peers on the value of usability testing and iterative design, you’ll probably have a better-than-even chance of being able to effectively use usability evaluation techniques to assess your design. Plus, taking a balanced approach, as I’ve recommended in Guideline 2, keeps you attuned to recognizing the delighters—the Wow, cool! moments—your design may engender.

Deborah Mayhew wrote eloquently about the benefits of designers’ being more sensitive to specific concerns and questions: “Personally, I believe there are distinct advantages to testing our own designs that probably outweigh the potential biases we may bring. As designers, we have concerns, questions, or reservations about specific aspects of our designs, and we design our tests around those. We are looking with a very focused eye to learn about those aspects. An independent tester will not be so sensitive to those aspects. I realize this can introduce bias or at least risk missing other things we are not looking for. But I agree with [some of the other commenters] suggesting that, for many of us, we really are pursuing the truth and looking for the problems in our designs rather than simply trying to confirm them. I guess, as with many things, it depends on the integrity of the practitioner, but personally, I would always rather test my own designs than leave it in the hands of others who are not intimately familiar with what I am trying to find out about my design.”

I wouldn’t personally go as far as Deborah did when she said she would always rather test her own designs. But her point about the integrity of the designer really resonated with me and here’s why: When we design user interfaces, we enter into an implicit agreement that everyone who has designed a user interface has entered into—whether consciously or not. We are essentially agreeing to try to solve the design problem to the best of our ability.

The object we’re designing could be a Web form, an electric toothbrush, or a suspension bridge. It doesn’t matter. When we design, we are in effect agreeing to honor the craft of design. And honoring the craft of design, indeed, takes integrity. What separates craft from art—an inherently subjective endeavor—is that, in almost all instances, you can assess it objectively. (Although, to be fair, it’s not always clear what measures to use when assessing a design’s goodness of fit to a design problem—or even what the problem is, for that matter.)

All of this brings me to a third guideline, which is this:

Guideline 3—When you’re trying to solve a design problem, usability testing serves design. It’s a tool. Use it to improve your design, not to justify your actions.