The Elements of Interaction Design

By Dan Saffer

Published: May 8, 2006

An excerpt from Designing for Interaction: Creating Smart Applications and Clever Devices. ©2006 New Riders.

“For interaction designers, who create products and services that can be digital (software) or analog (a karaoke machine) or both (a mobile phone), the design elements are more conceptual.”

Other design disciplines use raw materials. Communication designers use basic visual elements such as the line. Industrial designers work with simple 3D shapes such as the cube, the sphere, and the cylinder. For interaction designers, who create products and services that can be digital (software) or analog (a karaoke machine) or both (a mobile phone), the design elements are more conceptual. And yet they offer a powerful set of components for interaction designers to bring to bear on their projects.

Motion

In much the same way that inert gases don’t mingle with other gases, objects that don’t move don’t interact. An interaction is some sort of communication, and communication is about movement: our vocal cords vibrating as we speak, our hands and arms writing or typing as we send email or instant messages, sound and data moving between two entities.

“Interaction designers are very concerned with behavior: the way that products behave in response to the way that people behave.”

We communicate in many ways and through many different products, from mobile phones to email. Those products and the people who use them generate behavior, and interaction designers are very concerned with behavior: the way that products behave in response to the way that people behave. And all behavior is, in fact, motion: motion colored by attitude, culture, personality, and context. There’s wide variation even in such universal and seemingly simple behaviors such as walking (that’s why, for instance, there’s a need for both high-impact walking shoes and walkers for the elderly), and the designs we create have to understand and account for those variations in motion. Even a simple motion like pressing a key on a keyboard can be difficult if you are elderly or infirm.

Motion is often a trigger for action, as when your finger clicks the button on your mouse. The triggered action (or at least the feedback for that action) is often about motion as well. You click a Web site link, and the page changes. You press a key, and an email window closes. There is motion on your screen.

Without motion, there can be no interaction.

Space

“Interaction designers work in both 2D and 3D space, whether that space is a digital screen or the analog, physical space we all inhabit.”

Movement, even on a subatomic level, happens in some sort of space, even if the boundary of that space (as with, say, the Internet) is unclear. Interaction designers work in both 2D and 3D space, whether that space is a digital screen or the analog, physical space we all inhabit.

Most often, interaction design involves a combination of physical and analog spaces. You make a gesture in physical, analog space—for instance, turning a knob on your stereo—and you see the results on its digital display screen. The reverse can, of course, be true as well. You can stream music from your computer through your stereo and into physical space.

Most interaction designers underutilize 3D space on screens. The physical flatness of our monitors and display screens causes us to ignore what the Renaissance painters discovered so long ago: perspective. Objects, even in a 2D space, can appear to move backward and forward in 3D space. Perspective creates, alongside X (height) and Y (width), a Z (depth) axis on which to work. Web sites are notably bad in their use of Z space.

Starbucks® cafes typically make excellent use of physical space, with the ordering area separated from the fulfillment area where customers receive their beverages and those areas separated from the area where people can customize (add milk and sugar and other condiments to) their drinks. Compare that to the typical crush around a single counter of a fast food restaurant.

Space provides a context for motion. Is the action taking place in a quiet office in front of a computer screen or in a crowded, noisy airport in front of a kiosk?

All interactions take place in a space.

Time

“Interaction designers need an awareness of time. Some tasks are complicated and take a long time to complete—for instance, searching for and buying a product.”

All interactions take place over time. Sometimes that time can be near-instantaneous, like the time it takes to click a mouse. Sometimes it can involve very long durations. You can still find online usenet messages (usenet is a sort of bulletin board system) from decades ago.

Movement through space takes time to accomplish. As every gamer will attest, it takes time to press buttons (around 8 milliseconds at the fastest). Even with broadband speeds, it takes time for packets of data to travel from distant servers through the physical wires and perhaps through the air via wireless signal to your computer.

Interaction designers need an awareness of time. Some tasks are complicated and take a long time to complete—for instance, searching for and buying a product. Many e-commerce Web sites require you to log in before purchasing, and that login session will be active for a set time. Imagine if Amazon® or other e-commerce sites timed out every few minutes and required you to log in repeatedly while shopping—it’s unlikely you’d buy much from them. Some travel and concert-ticket Web sites make users race against the clock to enter their credit card information before their selected seats are lost.

Digital time is definitely not human time. Digital time is measured in milliseconds, a single one of which is considerably shorter than the blink of an eye. Changes done by the computer can be so instantaneous that programmers need to program in delays so that humans can detect them.

You can feel the impact of milliseconds, however. Extra milliseconds added to every keystroke or mouse-click would probably make you think your computer is slow because of the tiny delay. Several hundred milliseconds would cause frustration and anger, and a single-second delay each time you pressed a key would probably make your computer unusable.

Time creates rhythm. How fast something pops up on the screen or how long it takes to complete an action like renewing your driver’s license controls the rhythm of the interaction. Games are often about rhythm: how many aliens come at you at any given moment, or how long does it take to complete a level. Rhythm is also an important component of animation: how quickly does a folder open or close on the desktop, how slowly does a drop-down menu slide open. Interaction designers control this rhythm.

Battery life (the duration of which is slowly getting better) is another element of time of which designers need to be cognizant. Some features, such as a backlight, drain more battery power than others and thus decrease the amount of time the device works. A mobile phone that worked for only 10 minutes unplugged from power wouldn’t be of much use.

Interactions happen over time.

Appearance

“How something looks gives us cues as to how it behaves and how we should interact with it.”

How something looks gives us cues as to how it behaves and how we should interact with it. The size, shape, and even weight of mobile devices let us know that they should be carried with us. The sleek black or silver look of digital video recorders like TiVo® devices tell us that they are pieces of electronic equipment and belong alongside stereos and televisions.

Appearance is the major source (texture is the other) of what cognitive psychologist James Gibson, in 1966, called affordances. Gibson explored the concept more fully in his 1979 book The Ecological Approach to Visual Perception, but it wasn’t until Don Norman’s seminal book The Psychology of Everyday Things, in 1988, that the term spread into design. An affordance is a property, or multiple properties, of an object that provides some indication of how to interact with that object or with a feature on that object. A chair has an affordance of sitting because of its shape. A button has an affordance of pushing because of its shape and the way it moves (or seemingly moves). The empty space in a cup is an affordance that tells us we could fill the cup with liquid. An affordance (or, technically, a perceived affordance) is contextual and cultural. You know you can push a button because you’ve pushed one before. On the other hand, a person who has never seen chopsticks would be puzzled about what to do with them.

Except to the visually impaired (for whom texture often substitutes), appearance also conveys emotional content. Is this product whimsical or serious? Practical or playful? Appearance can also convey other attributes that may be meaningful: Is the object expensive or cheap? Complicated or simple? Daunting or approachable? Single use or enduring? Structured or casual?

Appearance has many variables for designers to alter

  • proportion
  • structure
  • size
  • shape
  • weight
  • color (hue, value, saturation)

All of these characteristics (and more) add up to appearance, and nearly every design has some sort of appearance, even if that appearance is a simple command line.

Texture

“The sensation of an object can provide clues as to how it is to be used….”

While texture can also be part of the appearance, how an object feels in the hand can convey the same sort information as appearance. Texture, too, can convey affordances. The sensation of an object can provide clues as to how it is to be used as well as when and where. Is it solid or flimsy? Is it fragile or durable? Do the knobs spin or push or both?

Texture can convey emotion as well. A fuzzy plush object conveys a different meaning than a hard metallic one.

Designers can also work with texture variables such as vibration and heat to signify actions. A mobile phone can vibrate when a new message arrives, and one could imagine it growing colder the longer it’s been since a voice-mail message arrived.

Sound

“Sound is underutilized (some would say rightfully so) in interaction design, but even a little bit of sound can make a major difference in a product.”

Sound is a small part of most interaction designs, but it can be an important part, especially for ambient devices and alerts. Sounds possess many variables that can convey information as well. You wouldn’t want a loud screech to come out of your computer every time you received email, and a soft whisper wouldn’t exactly cause traffic to move aside for an ambulance.

Sounds are made up of three main components, all of which can be adjusted by a designer:

  • pitch—How high in range a sound is. Is it high pitched like a bird’s song or deep like thunder?
  • volume—How loud a sound is.
  • timbre or tone quality—What type of sound it is. Sounds played at the same volume and pitch can seem very different. Think of a middle C played on a trumpet and one played on a piano.

Sound is underutilized (some would say rightfully so) in interaction design, but even a little bit of sound can make a major difference in a product. Steve Jobs insisted that the wheel on an iPod® make an audible click that could be heard without headphones.

All of these elements of interaction design comprise any interaction designer’s toolkit, and while interaction designers may not consciously manipulate them, they are the building blocks of interaction design.

17 Comments

Isn’t that all kinda obvious?

They are the basic building blocks, the same way that a line is for graphic design. They may seem obvious, but one of the things missing in the discipline was a discussion of what gets manipulated by interaction designers to create their work. This was my answer.

I think that intentionally breaking things down to elements is not an obvious task at all. But the question you are probably thinking is why even bother? Is this going to help my practice?

It all depends on your style and interest. I can see something like this being used in a design studio curriculum where in a studio class these elements are explored as key ingredients. Just as by changing ingredients, heat levels, and length of cooking you can end up with completely different meals (let alone cooking surfaces).

I can see these elements being mixed and changed and skewed toward learning to master them as ingredients in a designer’s kitchen. This is a common practice with Graphic Design and Industrial Design where an entire studio might be about learning how color change alone effects the results of your design.

In my own day-to-day practice, I find that it is very important for me to intentionally imagine these or similar elements as I work on my designs.

Now some people may not be as analytical as myself and won’t get use out of this, but I do believe there are both academic and practical value propositions for looking at the above.

On a related, yet separate note, I wrote a piece expanding on where Dan left off from the above. I’ve been thinking a bit about “foundational” elements of IxD for quite some time, and I am happy that Dan is doing the same level of thinkig here. Here is my link: http://synapticburn.com/comments.php?id=143010C

What about taste? I want taste!

I agree with you about the underutilization of the Z-axis, which can be employed to show not only distance but time—for example, use perspective to show the user is starting a journey that will end at some point off in the distance.

Not a dimension, but perhaps an element: emotion. Also complexity. It’s interesting how these interact—for example it’s been shown that too many choices can result in a lessened feeling of well-being (and increased cognitive load). You could go on and on…

A nit or my mis-reading? :) “Space” section, paragraph 2: “Most often, interaction design involves a combination of physical and analog spaces. You make a gesture in physical, analog space—for instance, turning a knob on your stereo—and you see the results on its digital display screen.” Should it be: “Most often, interaction design involves a combination of physical and digital spaces.” My reading of the space section equates physical space as analogous to the analog world. Thanks, great article!

Jay, you are right, that’s a typo. It should read physical/analog and digital spaces. You make a gesture in a physical space and something happens in a digital space. Good catch!

Roy, I do go on and on about those things. I wrote a whole book, in fact! ;)

A gentle comment—After I read this, I wished I knew a little more about what the book would be about. The general concepts, or building blocks, of IxD are interesting, but I’d like to know more about how this book or these concepts could help me, specifically and pragmatically, with examples, in my day-to-day work as a designer. That may not have been the point of this article, but that’s what I was left feeling. Thanks!

Jim, feel free to check out the book’s Web site:

http://www.designingforinteraction.com

Ignore the ugly design for now—I’m fixing it!—but you’ll find a table of contents and some interview excerpts from the book.

This excerpt is from the more, shall we say, theoretical part of the book. There are also more tactical chapters on user research, documentation, interface design, and service design.

Breaking them down is fine in a “Semiology of Graphics” sense. The interesting issues are determining the lexicon and how these elements interact. In the two dimensional sense, we need a Wolfgang Weigart of interactive. And, what about the most obvious of all: reaction?

The variables discussed are no different than film, with the exception of texture. It isn’t interactive unless an interaction is specified and acting with sound can induce texture. How is this determined?

Nice post!

BTW, an example of reaction or behavior would be Disneyland. As an interaction system it is top-notch. The behavior would be characterizations and elaborations such as cast interaction: happy. A result of elaborating on this, or creating rules, would be the cast manual. Do this… Don’t do this…

Wonderful discussion and article. I look forward to reading the book—reading it, not “scanning”. I agree with Roy Zornow’s comment about emotion and complexity. I’m certain this is covered in the book, but there is the issue of unconscious behavior and motivations to consider. Unfortunately, they happen to be among the most difficult to predict, quantify, chart, etcetera, because they are the elements of our humanity.

Who we are—the majority of us being people—seems as important as anything else. I’ve long believed that we are actually in the “people business” in addition to technology, design, etcetera. Understanding people belongs in that useful toolkit. Thanks for letting me share.

What are the elements of conceptual design?

Michael, I do touch on both emotion and complexity in the book. Generating emotion through tools like mood boards and being aware of the context of use. Complexity is given a richer treatment, covering not only Tesler’s Law of the Conservation of Complexity, but also how to make things smart and clever by managing complexity.

Myleene, I’m not sure what you mean by conceptual design.

Negative space is not a separate element, but a property of each element when no detectable properties exist. In most disciplines, this absence has become as important as the elements themselves. To graphic designers this is known as white space and is treated as delicately as the type they set. Just as space has negative space, motion has stillness, and sound has silence. It is with this absence of an element that we are able to create rhythm, balance, and hierarchy.

Dan, great work.

As a storyteller, I appreciate the detail around time and the multiple perspectives woven richly into this post.

My experience includes personas and scenarios that are different from most. I allow people to wear many persona masks in my analysis. This provides a simplicity and flexibility you may find helpful. Scenarios are often used and almost always abused by being treated as interactions or workflows—sensitivity analysis tools or worse—rather than a simple combination of unique stories. Accountants, oddly enough, get it right. Though they don’t recognize scenarios when they create forecasts. Consider that at the core of the numbers in a forecast is a bunch of stories. If you have two forecasts, generally it is much more than sensitivity analysis and can be described as two unique sets of stories in the same way as most strategic scenario analysts would work. Google Pierre Wack for scenario analysis history.

I also swap product for expertise when I combine personas and scenarios, in order to understand expertise gaps moving forward in time. In this case, I reverse the polarity by creating an ideal persona based on future products and the behaviors and stories served. Once this is done, I compare the persona to the expertise pool to determine the variances. This can be done across the value stream, either to support HR internally or competitive intelligence externally.

Here are links to my blog posts:

NeuroPersona

Persona Behaviour

Numbers, Stories, and Personae

Cheers, Nick

Join the Discussion

Asterisks (*) indicate required information.