Designing for Touch

Pinch and Spread

This duo typically shrinks and enlarges images, maps, and Web pages. It’s a pleasingly immediate interaction that lets you grab an object and then crunch or stretch it.

This literal zoom effect is supplemented in a growing number of applications by a more metaphorical version called semantic zoom—an emerging convention thanks to its widespread use in Windows. There, semantic zoom zips between two views: a close-up and a bird’s-eye perspective on the high-level organization. For example, in the Zappos shopping app for Windows, the zoomed view shows all the departments with their product categories: hats, gloves, and so on, in Accessories, as shown in Figure 4.1. Pinch the browsing view of the Zappos app to trigger semantic zoom in Windows for a more visual overview of the store’s departments. For faster scanning, pinch that view to pull back to a simplified list of the same departments without their categories. Spread or tap a department to zoom back in.

Other approaches extend semantic zoom to navigate more deeply into the information hierarchy. For example, Photos for iPad offers pinch and spread as an alternative way to navigate between an album and individual photos, as shown in Figure 4.2. Pinch a photo in iPad’s Photos app to close it and return to the parent album’s thumbnail view. The gesture provides an alternative to tapping the Moments back button at the upper left. When you’re admiring one of your pictures, you can tap the back button to return to a thumbnail view of all the photos in the album. But you can also pinch the screen to return to that thumbnail album view. Here semantic zoom is deployed to let you move up and down the app’s organizational levels. Pinch a detail view—the photo—to close it and return to the level above—the album—or from the album view, spread a thumbnail image to open it to its detail view.

Double-tap

Like pinch and spread, double-tap zooms in and out. (Android adds nuance to double-tap zooming with double-tap and slide. When you slide up and down after a double-tap in Android, you can control the precise amount to zoom; sliding up zooms out and sliding down zooms in.) Double-tap has few conventional uses beyond zooming, however, making it ripe for experimentation in other contexts. BostonGlobe.com, for example, lets subscribers double-tap a headline to save the article for later reading.

You can count on your audience to understand these six gestures without additional help. But while reliable, this kit remains primitive—it simply ports existing mouse-and-cursor interactions to the touchscreen. These gestures are exactly as expressive as a mouse cursor, reducing the hand’s subtlety to a single jabbing finger. As a result, they tend to reinforce old, troubled desktop metaphors.

The Trouble with Buttons

Buttons have served us well in both the physical and digital worlds, but their translation to touchscreens is more unwieldy: buttons take effort, add complexity, and insert an abstract layer between you and the content. Touch has the potential to sweep away the abundance of buttons, menus, folders, tabs, and administrative debris we’ve accumulated over decades of desktop computing. A new choreography of gestures can and should replace those timeworn controls to let us work directly with content.

Buttons Take Effort

Physical interfaces require physical effort. For small touchscreens, that effort is modest, usually only the sweep of a thumb. As screens get larger, however, the effort increases. Roaming the screen means shifting your whole hand, or even your arm, to work the controls. I know, I know—how hard is it to flip your hands across a screen? But fatigue sets in with time and repetition. A few years ago, I judged a digital magazine competition, poring over hundreds of iPad apps. After several hours of bad ergonomic designs, no kidding, my arms were sore from reaching over and over into the screen for basic navigation tasks. Call it iPad elbow.

Consider the Back button in the top left of many iPad apps. We’re asked to hit it all the time—to go back, to browse an app’s hierarchy, and so on. The button is in the thumb zone, but it still takes a moment of concentration and effort to hit it. Despite the sweeping expanse of a tablet screen, this tiny patch of pixels demands constant attention.

On larger screens like tablets, favor coarse gestures over fine-tuned pecking. Let people paw at the whole screen for basic actions like simple navigation. In iPad’s Mail app, shown in Figure 4.3, for example, the Inbox back button on the left requires a slight reach to open the message drawer, but you can also pull it open by swiping left to right anywhere on the screen, which lets you access the same content no matter where your hands sit. The entire screen becomes the control—no special trip to the back button required.

Coarse gestures help reduce errors and improve accessibility. When Boeing designers asked me how they might make touch interfaces more forgiving of pilots’ errant fingers during turbulence, I suggested using coarse gestures—for example, a swipe, full-hand pinch, and so on—to let pilots slap at the screen instead of requiring careful button presses. The same advice holds for seniors, children, or others with limited motor control. Likewise, these gestures provide no-look control for those who have limited vision of the screen—like drivers and cyclists.

Big gestures also tend to develop into reflexes. Traditional interfaces rely on visual memory, asking us to scan buttons and labels to absorb their meaning. Touch interfaces ask some of the same, but blend in muscle memory—a subconscious knowledge of the interface that seems to spring right out of our hands and fingers. Like playing an instrument or typing a keyboard, the repetitive actions of working a touchscreen settle into instinct. Touchscreens depart from physical interfaces in an important way, though: their glass surfaces don’t provide the physical feedback of an instrument’s strings or a keyboard’s buttons. Fine controls slow us down by demanding we look at the screen. Coarse gestures, however, quickly embed into muscle memory and require little visual processing; users of laptop trackpads think nothing of the two-finger gesture for scrolling, for example. Broad expressive actions make the best touch interfaces more like playing an instrument than using a push-button tool.

Buttons Add Complexity

We’ve all been baffled by the dense undergrowth of buttons that blanket our cars, home appliances, remote controls, and other everyday machinery. On a recent summer trip, my family counted over eighty buttons in our Citroën rental car; it took us ten minutes to figure out how to get the thing moving. (The Citroën C4 has a steering wheel with thirteen buttons and four scroll wheels—scroll wheels!) This complexity of common interfaces mirrors the growing complexity of the associated device, creating a familiar design challenge: more features seem to call for more controls. But if you’re not careful, buttons start sprouting like mushrooms all over your interface.

Take a lesson from console gaming. What began as a single-button joystick in the first Atari games has evolved into elaborate, button-addled controllers. A standard Xbox One controller features eleven buttons, two triggers, two joysticks, and a D-pad. As shown in Figure 4.4, the first generation of iPhone games in 2008 ported over this button-based system, with awkward results. Onscreen buttons took up valuable space, obscured gameplay, and locked fingers and thumbs in place. The game Earthworm Jim ported console-style controls to small-screen phones, choking out the actual gameplay. The buttons were hard to use too: since fingers skid across glass, virtual buttons lack the reassuring no-look feel of their physical counterparts. The buttons backfired.

Game designers needed a new model—so they ditched buttons. With fewer controls, game designers pared down features. Simple but satisfying games like Angry Birds ruled touchscreens with one or two gestures. The nature of the game adapted to the nature of the input. As gesture gaming found success, more sophisticated games developed to rival cinematic console games. Some, like the fantasy action game Infinity Blade, used gestures to drive familiar hack-and-slash games. Others took an even more novel approach, creating gameplay purpose-built for the touchscreen. Games like République or Monument Valley invite you to tap inside the game environment to move the game’s hero. Instead of depending on complex button-based puppetry, they draw you in by interacting directly with the game world. It’s a shift in perspective that delivers the complex experience of a console game without the need for correspondingly complex controls.

In République, shown in Figure 4.5, you’re a hacker who guides the hero to escape from a mysterious facility. You view the world through surveillance cameras and tell her where to go and when to move. Most actions involve tapping on cameras to take them over, and then tapping on locations to tell the escapee where to go. The world itself is the control, no buttons required.

All software genres should explore a similar perspective shift, embracing more direct interaction and fewer buttons and controls. It’s worth acknowledging that touching a button is a kind of direct interaction. The trouble is that this connection is with a button, not the information you seek to manipulate. Or, put another way…

Buttons Are a Hack

Don’t get me wrong: hacks—and buttons—aren’t all bad. We invented electrical switches and buttons over a century ago to control objects at a distance. These switches were designed as messengers to carry our intent—for example, turn on the lights—to its destination—the light bulb. While convenient, this interaction is also indirect, completely disconnected from the thing we want to affect. A light switch here for a light bulb over there isn’t obvious; it has to be discovered, learned. When I’m in a new hotel room, it takes me a minute or two of trying switches to figure out how to turn on that one light. But that’s far better than stumbling into a dark room with a ladder and climbing up to screw in a bulb. The light switch is an inspired hack. When it’s not convenient to interact with the primary object, adding a control to work it from a distance is ingenious.

Buttons are workarounds for moments direct interaction isn’t possible. It’s a similar story for buttons in virtual interfaces. We created buttons and tabs and sliders as skeuomorphic intermediaries to work with digital information and actions that were beyond reach or easy representation. Buttons still have their place, and their obvious labels and clear calls to action make them especially useful. Embrace them as needed, but recognize that they remain workarounds. Every time you add a button to your layout, challenge yourself: can I find a way to manipulate content more directly?

Information as Physical Object

Every digital interface is an illusion, a thin layer of magic atop a churn of ones and zeros. For the first time, though, touchscreens present the opportunity to create the illusion that there is no illusion, that there’s nothing between you and the content. This book has stressed the importance of physical interaction; to complete the illusion, let’s now apply that same thinking to the data itself.

Reimagine your information as a physical object. Ask this of every element of your interface: “What could I do with this piece of data if I could slide, stretch, and poke at it under glass?” Semantic zoom—bestowing pinch-to-zoom physicality on the underlying information architecture—is one example. Now consider how you might select a date range. A typical design uses two calendar-style date pickers, a solution that does use physical metaphor by riffing on paper calendars. But that doesn’t imbue data with physical properties and direct interaction. Instead, as shown in Figure 4.6, you can imagine the date range itself as an object with mass and elasticity—a rubber band whose endpoints you can squeeze or stretch to whatever size you want.

Make Content the Control

This reimagining helps clear away as much interface as possible between the user and the content. All UI is social convention, and those conventions run into trouble when they’re not evenly understood. In his book Living with Complexity, designer Don Norman notes the holes that distinguish salt and pepper shakers—and how people have decidedly mixed opinions on whether a single hole denotes salt or pepper. Norman points out that it doesn’t really matter which is correct. What matters is what the person who fills them believes. That would be fine if everyone understood the system the same way, but that’s not the case. Even if I’m certain that salt should be in the shaker with one hole, I can’t trust that others share that view; when I’m at a restaurant, I test by shaking a little into my hand first. I don’t have confidence in the system.

As designers, we’re the ones filling the shakers. Our job is to give users confidence. To do so, we often fall back to explicit labels for Salt and Pepper. But that still requires visual processing and command of English to decipher each shaker. You know what’s better? Glass bottles that let us see the salt and pepper inside, as shown in Figure 4.7, no reading or scanning of labels necessary, just grab what we need.

Image sources: Joe King and Black Country Museums

Touchscreen photo galleries are nearly perfect examples of this. They’re very dense interfaces, yet with almost no controls. It’s all content: tap a photo to make it bigger, then swipe through the collection. Interaction is tied entirely to the content; the information itself is the interface. Marshall McLuhan famously said, “The medium is the message.” When we create the illusion of direct interaction with information, we can finally say that the message is the medium.

So what does interaction look like when we press this message under two-dimensional glass? In the physical world, we have a word for individual pieces of flat content; we call them cards. That’s why all the major touch operating systems use cards—or tiles or panels—as their core metaphor for representing direct-interaction content.

The Power of the Card Metaphor

Cards have become a popular way to represent individual data objects: a photo on Facebook, a flight from TripIt, a contact, a coupon, a Yelp review, a Google Now reminder, tweets on Twitter, a game level, and so on. A variety of services embed cards as tiny multimedia canvases into apps, Web pages, social streams, notification windows, and more. Figure 4.8 shows cards containing pins on Pinterest, on the left; on TripIt, on the right.

Until recently, we settled on sharing those chunks of information via URLs in email or text. Now, data cards provide a snack-sized, portable format that shuffles neatly into interlocking modules in big-screen responsive Web sites, or as the main event in small-screen apps. Plus, cards are fun: when we represent data objects like trading cards, business cards, or paper coupons, we have a natural and even nostalgic impulse to share and exchange them in the same way—collect ’em all!

Cards also suggest lots of physical interactions. The most basic and familiar, of course, is flipping. On phones, screens are stacked just like a deck of cards. You swipe back and forth through your browser history, flicking with your thumb as if you were dealing cards at your weekly canasta game—it’s effortless, a wonderful coarse gesture.

Swap cards for pages, and you get the Web’s longstanding metaphor. But even the first generations of touchscreen browsers forced you to flip through these pages by tapping a button, confusing the metaphor. When was the last time you turned a paper page by hitting a button? Most touchscreen browsers now do the right thing, letting you swipe through page history. After decades of using a physical metaphor—the page—to describe the Web, we’ve finally arrived at a physical interaction to match.

This reconciliation is the crucial work of touchscreen interaction design. Embrace the verbs that obviously match your interface’s nouns. Consider everything you can do to a card in the real world: flip, fold, shuffle, stack, turn over, stretch, sort, crumple, dog-ear, throw aside. All those physical actions are available as a springboard for your interface metaphor. What does it mean to flip over a data object, to stretch it, to crumple it?

Facebook created an app called Paper to play on this card-like physicality, revealing a uniquely touch-friendly way to explore the Facebook timeline. Facebook Paper imagines its data as a physical card, where every physical action has a corresponding data action. Web-headline cover cards unfold to reveal their interactive insides. Everything is represented as a card. The app is organized into feeds, represented as decks that you can sort, shuffle, or discard to create new collections. Each item in a feed is a card that you can flip through. When a card features a Web article, you swipe up to unfold it like a newspaper and start reading, as shown in Figure4.9.

The gestures in Facebook Paper have a natural logic once discovered, but they’re not always obvious to the newcomer. Helping your audience find, learn, and adapt to these new gestures is a major challenge—one we’ll tackle in the next chapter. That said, when gestures hinge on simple interactions based on the physical world, you may not have to do much education. As we embark on designing interactions beyond the basics, let’s begin by looking around us for inspiration.

Let the Real World Be Your Guide

To make sense of the world, we put our trust in a mix of physical laws and human conventions. Gravity has proven to be awfully reliable, for one, and so have many of our own social constructions—salt and pepper shakers notwithstanding. Screws turn clockwise, the pages of Western books proceed from left to right, a checkmark signifies a completed task, and red means stop. These things set our expectations and shape our behaviors as we move through the physical world. Apply those expectations and behaviors to your touchscreen interface, and you give your users a familiar and predictable experience. Here are a few strategies to do so.

Borrow One-to-One Interactions

The most no-nonsense approach is to interact with the screen exactly as we interact with another real-world object. Sketchpads are a straightforward example, creating the illusion—again!—that the touchscreen is interchangeable with a piece of paper. (There’s a reason we call these things pads and tablets.) Take Paper—an iPad drawing app not to be confused with Facebook Paper—which has gone to remarkable technical lengths to recreate ink on paper. Choose among pencils, pens, or nibs and draw with a stylus or your finger. Freeform painting becomes the app’s primary gestural interaction—the same action that works in the real world also works on the tablet—simple.

Often, though, you can borrow a familiar physical interaction without copying the entire original artifact. You know how a knob or dial works: cranking clockwise means more or forward; counter-clockwise means less or backward. But if you want to use that crank motion, you don’t have to add an actual knob to your interface. Instead, use the action as inspiration for a gestural equivalent. The Paper app does this well: cranking two fingers counter-clockwise anywhere onscreen performs the undo action, traveling back in time to remove strokes from your drawing. Change your mind? Crank clockwise to redraw your strokes, as shown in Figure 4.10. As you borrow from the real world, remember that it’s the physical action you seek to import, not the original object; it’s about the crank, not the knob.

Lean on Established Symbols or Notation

As we translate physical cranking motions to screen gesture, we transform them into notations, a meaningful shorthand. These symbols are human inventions—clockwise and counter-clockwise doodles have no inherent meaning in nature. But again, all user interface is social construction; lifting from well-known conventions makes your interface feel instantly familiar, intuitive.

Sometimes entire notation systems can be borrowed whole cloth. The specialized set of proofreaders’ marks, for example, is at once expressive of complex ideas and universally understood among editors, as in Figure 4.11. An application aimed at that audience might adapt that notation into shorthand gestures to delete, move, or insert new paragraphs.

Adobe Comp is a wireframing app for iPad. Its touch-friendly interface piggybacks on common wireframe symbols, borrowing the sketch notation of wireframing. In Figure 4.12, drawing a big X adds an image placeholder and scrawling a stack of lines inserts a column of text. Erase an element by scribbling it out. The app converts these symbols into wireframe components and, when you’re done, exports to InDesign, Illustrator, or Photoshop—casual whiteboard-style input transformed into formal wireframe output. Now, this may not be any more efficient than producing a wireframe with Microsoft Visio or OmniGraffle on a desktop, but it’s the quickest way to do it on a touchscreen. These fluid, sketch-like gestures create layouts much faster than a marathon of taps through a series of desktop-style control panels.

Apply Physics to Digital Objects

Adobe Comp borrows directly from how we work a familiar interface—paper—but you don’t have to be so literal. You can also apply real-world physics to digital objects by giving them a sense of mass, of physical presence. For instance, pinch-to-zoom lends squeeze-n-stretch physicality to photos or maps. The way a scrolling screen bounces when you slam into the end of content adds solidity to intangible data. The iPhone app Clear, shown in Figure 4.13, has an all-gesture interface that relies on simple physicality. To insert a list item, spread two items apart to make room. Clear treats to-do-list items like physical building blocks that you can squeeze, slide, or nudge aside, and those actions have semantic meaning for the data:

• Insert a new list item by spreading your fingers between two other items to make room.
• Mark an item complete by pushing it aside with a swipe.
• Pinch a list to close it.

These are simple gestures for simple physical actions, mapping naturally to how you might rearrange list items if they were arranged physically on your desk. When virtual elements behave with such familiar physicality, our brains flow naturally into the interaction.

Honor Physical Constraints

If things we can do in the physical world inspire gestural interactions, so too do things we can’t. TouchUp is an iPad app that draws filters or effects on photos. The simplest example is painting a color on the photo, using your finger as a brush. But what if you want to change the brush size? That’s easy, right? After all, desktop apps always address this by offering either a slider or a brush palette to choose a new brush size. Thing is, you already have a brush—your finger—and it doesn’t change size. Changing the size of your finger’s impression on the screen to anything other than the size of your fingertip introduces uncertainty. You have no idea how large a mark you’ll leave. You shift from direct interaction to abstract guesswork.

So TouchUp doesn’t let you change brush size. Instead, you change the canvas size: pinch to zoom out and spread to zoom in. Your finger brush is always the same size onscreen, but because you’re drawing on a super-zoomed photo, the result is a fine line on the image.

It seems obvious when you see it in action, but it turns the traditional desktop approach on its head. When you deal with the physicality of touch, you have to rethink familiar solutions. With every solution, ask yourself: does the old way still make sense, or does direct interaction demand a new approach?

When you use the real world as your guide, you create interfaces that are instantly understandable. Some actions, however, are freighted with more meaning or complexity than can be easily packed into a simple physical action; they may require more abstract gestures. Those advanced power moves have a parallel in traditional computing.

Gestures as the Keyboard Shortcuts of Touch

Quick-to-discover gestures, combined with well-labeled traditional controls, should always form the foundation of your interface. Always make it easy to figure out your application’s basic actions. But don’t shy from deploying more abstract gesture shortcuts as alternatives alongside standard controls, like keyboard shortcuts on the desktop. Earlier, for example, I mentioned that the iPad Mail app supplements the back button with a coarse swipe. Or, from Chapter 3, take Vimeo, where a gesture inside a list view saves a trip to the detail page: swipe left or right across a video to share or favorite it. In both cases, the slow way is still available, but gestures are expert power moves.

These speedy shortcuts aren’t restricted to swipes and taps. More fingers multiply the possibilities. A five-finger touch might toggle between views of Sent mail and the Inbox. In a newspaper app, a two-finger swipe could fast-forward to the next section, not just the next page. A future of multifinger gestures promises a richer interaction language, where abstract gestures help shoulder complex actions.

The Unrealized Potential of Multifinger Gestures

The iPad app Uzu is “a kinetic particle visualizer,” but your first impression of it might be more like a toy to hypnotize stoners, an interactive lava lamp. In Uzu, different finger counts trigger different animations and actions, as shown in Figure 4.14. Touch the screen with one finger and sparks shoot off like fireworks. Add a second and sparks swirl between the two, while a third finger creates a vortex among all three. Change the hue and particle size by touching the screen with all ten fingers and swiping up and down, or left and right. As you get the hang of it, your fingers fly and dance across the screen, and it feels like playing a kind of Technicolor visual keyboard—more instrument than tool.

At first blush, this might not seem relevant to that financial services intranet app that you have to build, but Uzu offers useful lessons for all kinds of touch interfaces. In particular, this multifinger approach recalls the role of Alt, Command, or Function keys; ten fingers afford ten modes or actions. Fingers become function keys. Just as expert typists fly through words, or power users deploy keyboard shortcuts to fly through tasks, multitouch gestures likewise help us to move effortlessly through touch interfaces. These gestures, while abstract, allow power users to accomplish tasks more economically.

If multifinger commands possess such powerful potential, why don’t we have more of them? While phones have supported multitouch for years, they’re not great at it. One-handed grips and small screens have encouraged us to tap away with a single finger or thumb instead. Larger screens hold more promise. The size and weight of larger tablets requires you to use two hands or rest the tablet on your lap, so you always have at least one hand free—with a screen big enough to invite multiple touches at once. The same goes for hybrids and laptops.

Obstacles exist, though. Accessibility is a major one: not everyone has full mobility of hands and fingers—or even all their fingers, for that matter. For certain disabilities, a five-finger pinch is a non-starter. Discoverability is another stumbling block. How are we supposed to know when abstract actions like a three-finger swipe or a five-finger touch are even available? We’ll turn to strategies for revealing gestures in Chapter 5, but consider it best to treat these more abstract multitouch gestures as alternatives—expressive supplements to buttons and other traditional interactions. People should still be able to accomplish any action with simple taps and swipes, though it may take longer.

Navigating Heavy Gesture Traffic

As you layer more and more gestures into your interface to supplement or replace traditional controls, those gestures start to get crowded. For one, the operating systems and browsers claim key gestures so that your app or Web site has to scrabble among the leftovers; for another, gestures—especially coarse gestures—eat up lots of room and jostle for space on the screen.

Make Way for System and Browser Gestures

As a designer, you have to jockey with browsers and operating systems for key gestures—and if the system gets there first, it wins. Remember Android’s screen-bottom system buttons in Chapter 1? The operating system has first claim, so app designers must make way, taking their controls to the top of the screen. The same goes for system gestures; app designers have to work around the operating environment to avoid gesture conflicts.

iOS for iPad, for example, offers coarse gestures to move between apps. Swipe left or right with four or five fingers to switch among recent apps, or pinch with four or five fingers to close an app and zip out to the home screen. You can just slap at the whole screen to navigate apps—exactly the kind of coarse gestures that a tablet OS should embrace. I’m a huge fan of the spirit of these gestures, but I’m not wild on the execution. If only Apple had followed the interaction already adopted by other platforms, like Windows and more than a few defunct touch operating systems, including Symbian, BlackBerry PlayBook, and Palm’s WebOS. They all use edge gestures, a technique that is both more internally consistent and more deferential to individual apps. Edge gestures start on the frame, or bezel, of the device and swipe into the canvas. When used for app switching, this creates the illusion of knocking screens aside by shoving them at the edge.

Edge gestures match physical action with the conceptual metaphor of the OS. If you consider apps as the front-and-center canvas of the device, then the operating system is the frame that supports that canvas. When OS-level gestures start from the bezel, action matches expectation: this gesture works outside the current app. You’re working on the frame—the operating system—both physically and metaphorically.

In contrast, iPad’s app-switching actions work within the canvas itself, territory that should be dedicated to the current app. This creates confusing competition with app interaction: Will this gesture apply at the app level or at the operating-system level? Apple could have avoided this ambiguity by anchoring its gestures at the edge. By putting them inside the canvas, Apple swiped some great gestures from designers’ arsenals.

Browsers also claim a hefty share of useful gestures for themselves. Pinch, double-tap, swipe, long-press… These core gestures already have meaning in browsers, taking them off the table for Web designers. You could override those gestures by capturing their touch events and hijacking them for your own purposes—to make a long press trigger a custom behavior instead of showing a contextual menu, for example. But breaking a browser feature is almost never a good idea. Making browser gestures work inconsistently across Web sites undermines user confidence when we still need to shore up gestural interactions and establish standards. Let browsers have their gestures and work with what’s left.

Give Your Gestures Elbow Room

Managing gesture density means more than avoiding browser and OS conflicts; you also need to consider the physical space these gestures occupy, or they begin to overlap and collide. When gestures pile up, so do user errors. Consider a photo gallery that lets you swipe back and forth through photos inside a single Web page. On small screens, these swipes easily run afoul of browsers’ edge gestures; mobile Safari, for example, reserves edge-swiping left or right to cruise through your browser history. In our photo hypothetical, a wide accidental swipe while browsing the gallery would jump you out of the experience entirely, dumping you into another page from your history. Previous versions of Chrome for Android sidestepped this problem by limiting gesture real estate. A left or right swipe from the edge used to change tabs, but people kept switching them by accident while swiping within the page. So Chrome’s developers limited the tab-hopping gesture to a swipe across the address bar, instead of anywhere on the page. The move gave space and freedom to gesture-wielding Web designers. (Chrome finally gave up on tabs altogether in Android 5.0 Lollipop.)

Ease risky gesture density by anticipating it. That swiping gallery might use the HTML5 history API to update the browser history with each swipe. An accidental edge swipe backward in that scenario would return you to the previous slide—the same result as a regular swipe in the gallery.

Radial Menus Reduce Gesture Density

Sometimes old techniques resurface to solve new problems. The radial menu rolled out half a century ago but may have finally found its moment as a way to sidestep gesture conflicts. A radial menu is a set of options that spin out like spokes from a wheel. Microsoft’s note-taking app OneNote, for example, features a radial menu as a kind of right-click contextual menu, as shown in Figure 4.15. Tap the app’s ever-present menu icon, and out pops a wheel of actions to apply to your current selection. Drag your finger to the one you want and release.

At first glance, these menus might seem more complicated than a plain toolbar, full of visual information to process. At their core, though, radial menus are gesture-based: touch-swipe-release. That’s why some call radial menus marking menus: it’s like making a mark on the screen. Swiping to two o’clock has one meaning, and swiping to six o’clock another. You get faster over time, because radial menus take advantage of muscle memory in a way that list-based menus cannot. In iOS Messages, for instance, you trigger radial menus to text an audio clip, photo, or video, as shown in Figure 4.16. It’s a fluid motion that quickly becomes a habit. Tap and hold the microphone icon, and a radial menu pops up as the app starts recording; flick up to send the audio; flick left to delete; or let go to pause.

Another benefit is that radial menus keep their gestures very compact. They start from a specific point on the screen—ideally on the content that you seek to manipulate, though often on a menu button or other trigger instead. This fixed anchor point reduces errors by demanding extra care; it asks you to press on the element you want to affect before launching into the gesture. Radial menus are more precise and help bring order to otherwise crowded gesture interfaces.

Radial menus have been around since the late 1960s but until recently never got much traction in traditional mainstream interfaces, with one exception: games. Combat-based games use radial menus for speedy access to inventory or combat options, as in the kind-of-awesomely-named Game of Thrones: The Game, which uses a radial menu to control the action, as shown in Figure 4.17. It makes good sense that trigger-finger games have adopted the radial menu over a more typical list. In games, limiting interruptions is essential to the experience, and radial menus are more efficient than other selection tools.

The research on this has been in the can for over twenty-five years. A 1988 study did the comparison and found that for a specific test of eight-item lists, users were faster with radial menus than linear lists. And it turns out that speed only improves. That was borne out in a 1994 study by Bill Buxton and Gordon Kurtenbach, who tested radial-menu speed with a stylus. Over time, they found that expert users stopped looking at the menu at all, marking the screen with no-look gestures instead of pecking at buttons. Once they made that transition, selection became three times faster.

Like any technique, however, radial menus have their limitations too. Keep in mind the following caveats:

• They demand precision. \ While a radial menu’s fixed anchor point reduces gesture density, this is also at odds with the benefits of screen-roaming coarse gestures. Coarse gestures are ideal for navigation and basic controls, while radial menus are better suited for quick actions and tools.
• They don’t scale. You can only cram so many items around a circle. Eight seems to be the reasonable maximum. On smaller screens like phones, a radial menu gobbles up an especially big share of pixels, so it’s typically limited to three or four options.
• First use might be awkward. Despite the speed boost that comes with experience, we’re more at ease scanning down a list than around a circle. But that comfort level may not be so important when you look at actual use. “The effects of organization disappear with practice,” Buxton found in 1994. “Even when menu items have a natural linear ordering, selection using a radial menu is still faster and less error-prone than selection using a linear menu.” This fact, however, relies on rolling with one last constraint:
• Radial menus must be consistent. If you change the order or content of a radial menu dynamically, people fall back to visual selection, and you lose out on the muscle-memory speed boost.

On the whole, these limitations are modest—and actually help shape good use cases for radial menus, like primary navigation or consistent contextual menus. That’s exactly the role they’ve come to play in parts of Android, Windows, and iOS, as well as menu navigation for many popular apps—such as the iPhone apps for Yelp, on the left, and My Fitness Pal, on the right, in Figure 4.18.

Radial menus have been slower to arrive on the Web than those system and app environments, even though they’re well suited to both the medium and browser capabilities. Existing Web examples include the occasional jQuery plugin or a CSS3 clone of the Path app’s radial menu. Why aren’t we seeing more of these experiments? Truth is, it’s still painful to develop gestures in the browser. Let’s look at why.

The Heartache of Gestures on the Web

Some structural problems make designing browser-based gestures vexing, though not impossible. Browsers aren’t yet very good at meeting the interaction expectations that touchscreen devices have created, thanks to a couple of reasons in particular.

First, as we saw earlier, browsers already claim so many useful gestures for themselves. These gesture conflicts leave little to the designer beyond tap and swipe. (This is also why radial menus are a good fit for the Web: their tap-swipe interaction neatly uses the available combo.)

Second, JavaScript gives front-end developers only the most basic touch events: touchstart, touchend, and touchmove. It’s easy enough to detect a tap or maybe a swipe, but anything trickier gets complicated fast. Have fun coding a crank gesture, a two-finger rotation, or a multifinger swipe. Ideally, we’d have events for common gestures on any DOM element: pinch, long tap, swipe, rotate, and more. (Microsoft models this in its framework for building native Windows apps with HTML5, perhaps suggesting a way forward.) For now, we have to build them ourselves from scratch—or better, use a library like the excellent Hammer.js, which provides events for tap, double-tap, swipe, drag, pinch, and rotate.

Tools and techniques are emerging to help designers cope. Swipes are a particularly good place to start. They’re relatively easy to implement, and many sites already embrace swipe for next/previous navigation. For instance, you can swipe through Flickr’s photo galleries, next/previous articles at the New York Times, Google’s image search, and lots more.

You can venture beyond swipe, of course, but it’s harder work. Let’s examine what’s involved. The rest of this chapter introduces how browsers wrangle touch events and how you can use JavaScript and/or CSS to construct a few simple gestures. While we won’t sink into the gritty details—this isn’t a JavaScript tome—it’s important for designers to understand what’s realistic when coding for touch. Because coding touch events is never trivial, let’s start with when you don’t need them.

Stick with Click

As noted, most touchscreen browsers offer touchstart, touchmove, and touchend events. Tapping the screen also triggers good old-fashioned click, which allows swaths of mouse-focused code in legacy sites to get their jobs done in a touch environment. You can salvage your sanity by focusing on that single interaction model.

Whenever possible, stick with click. Despite the availability of more complex touch events, you don’t need to replace click events in your JavaScript unless you’re after something fancier than a tap. While we colloquially associate click with the mouse, it’s not strictly a mouse event. Instead, consider it a generic action: “I want to activate this element.” In most cases, if you want to trigger something when a user taps, capturing click does the trick. Skip the touch events and proceed as if you were coding for mouse.

Sticking with click has the added benefit of working across inputs. Despite its mouse-derived name, click is likely to remain the key action for Web browsers whether they’re driven by keyboard, speech, Kinect-style air gesture, or perhaps even a virtual-reality head twitch. Click is not only backward compatible; it’s future compatible too.

But click isn’t perfect. Much of this chapter has made the case that touch demands new approaches. Touch interactions vary in ways both obvious and subtle from mouse or trackpad events. Not least is the number of pointers to contend with. Mouse interfaces never have more than one cursor clicking away; with touch, you get the possibility of ten fingers—or even more with the help of friends—or toes. If you need to track more than one finger at once, click won’t cut it. Use click when you can, but switch to touch events when you need to do any of these:

• Track multiple touches—pinch, rotate, or two-finger swipe
• Take action while the finger is pressed to the screen—a touch-flavored mouseover
• Track finger motion—a swipe or drag

On this last item, you sometimes get some wiggle room. No-frills CSS has you covered for the most common swiping use case—galleries and carousels—so let’s begin there.

Use CSS for Gallery and Carousel Swiping

Sites often rely on complex JavaScript to create carousels and their associated swipes, but that over-engineers a surprisingly simple solution. If you instead use overflow:scroll in CSS, all modern touchscreen browsers will give you hardware-accelerated panning; swiping comes for free, no JavaScript required.

Consider a list of images:

Use CSS to display the list items inline in a horizontal strip, all set to a specific size:

And here’s the magic. Set the height and width of the containing ul list, and set its horizontal overflow—that is, overflow-x—to scroll. This tells browsers to scroll the images when they can’t all fit the available width. The result: touchscreen browsers display a horizontally swipeable carousel of images like that in Figure 4.19.

Who needs JavaScript? CSS and HTML are all you need to create a swiping carousel.

The last rule of the .carousel styles tells iOS to apply fancy momentum scrolling:

-webkit-overflow-scrolling: touch;

Now, a flick of the finger sends the page scrolling and it continues under its own momentum, easing to a stop for an effect that mimics natural physical behavior. Without -webkit-overflow-scrolling, the carousel would move only when you dragged it, with a hard stop, for a clunky, artificial interaction. Other modern touch browsers don’t need this hint and include momentum scrolling with no extra help.

A caveat: many older mobile browsers don’t handle overflow:scroll properly and instead treat it like overflow:hidden, which lops off any content that doesn’t fit. Instead of a zippy carousel, you’re left with one that won’t budge, putting any overflow out of reach. Happily, Filament Group has a fix: Overthrow is a JavaScript library that nudges these browsers to do the right thing, and throws in momentum scrolling to boot.

Add Snap Points to the Carousel

We now have a free-spinning carousel, which is great except that it can come to rest midway between images. Make the carousel snap to one of its panels when it stops scrolling by adding scroll-snap rules:

The scroll-snap-points-x rule tells the carousel to snap at the start of the first image—at 0px of the carousel’s width—then snap to every 400px from there, the width of every following image. At this writing, only Internet Explorer 10+ supports scroll-snap, and other browsers ignore it, scrolling fast and free without snapping. scroll-snap is on the standards track, however, so more browsers may adopt it in the future.

Tidy the Desktop Experience with Progressive Enhancement

On mobile and tablet browsers, elements with overflow:scroll are visually uncluttered. On desktop, though, these elements sprout scroll bars. Scroll bars are a fine starting point; they allow people to access your content, with a clear visual cue that more content is available. But scroll bars on mid-page carousels sure make for clunky visuals. Apply some carefully crafted JavaScript to enhance the experience. Use JavaScript to detect if the browser has touch or pointer objects and, if not, set the carousel to overflow:hidden and add next/previous buttons to move the .carousel contents left and right. I leave the details of coding this desktop carousel as an exercise for the reader, but first, a few notes. As we discussed in Chapter 1, detecting touch is not foolproof. For this specific widget, though, this is enough for a strictly aesthetic upgrade:

Because this touch detection isn’t airtight, this approach will inevitably give some touch browsers the next/previous buttons instead of swiping. Don’t fret. It will be a small minority of browsers, your content will still be accessible, and with chunky next/previous buttons, it will stay touch-friendly.

Touch Events in the Browser

As you’ve seen, a savvy mix of CSS and JavaScript click events can handle tap and swipe. If that’s all you need, our work is done; feel free to hop over to the next chapter. But if you need more complex gestures—multitouch, drag and drop, crank, rotate, and so on—it’s time to buckle up, put on your crash helmet, and code touch events. We won’t go far down the coding rabbit hole, but here’s a high-level overview of how touch events work.

The iPhone was the first popular platform to build JavaScript touch events into the browser, and other browser vendors quickly followed suit to provide compatibility with iOS. This approach became a W3C standard, and it’s now supported by nearly all modern touch browsers—except Internet Explorer, which has its own competing pointer model that may itself become a separate standard. We’ll look at that in a moment.

The prevailing touch-event model lets developers detect three events: touchstart, touchend, and touchmove. You may be familiar with their desktop cousins mousedown, mouseup, and mousemove, and these touch events work similarly: developers can detect when a touch begins, ends, or changes, and then trigger corresponding actions on the page. These events, like all JavaScript events, create an event data object that developers can access to get more information about the touch. Touch-event objects include three lists of touches, data objects that each refer to a finger or stylus currently touching the screen:

• event.touches—a list of all touch objects on the screen, not just the DOM element for this event
• event.targetTouches—a focused list of touch objects that includes only the touches on the current DOM element
• event.changedTouches—a list of the touch objects involved in the current event. In touchmove, for example, this list tells you which touches actually moved. Say that you’re pinching with your finger and thumb, but only your thumb moves. Then only that touch would be included here.

Each of the touch objects in these three lists in turn contains information about the coordinates of the touch and the target element that triggered the event. (If you touch a link, for example, the target element is that link’s <a> DOM element.) These event and touch objects enable developers to track the presence, position, and motion of fingers on the screen. For an introduction, check out the tutorial by Boris Smus, “Developing for Multi-Touch Web Browsers.”

Untangling Mouse and Touch Events

Earlier we covered how touch triggers a click event for backward compatibility, but that touch triggers a whole range of other mouse events too. Every time you touch and lift your finger to the screen, the browser fires off all of these events, in this order: touchstart; touchmove, if applicable; touchend, mouseover; mousemove, if applicable; mousedown, mouseup, click. This behavior is intended to make sure that sites coded for mouse-and-cursor interaction will continue to work on touchscreens, which is a good thing. However, a few tricky elements are worth calling out:

• Mouse events all happen in a flurry after the finger lifts off the screen. So touchmove doesn’t happen at the same time as mousemove, and mouseover is triggered when the finger isn’t even on the screen anymore. While mouse events remain available, in other words, they don’t match touch behavior one to one.
• Because a touch triggers both touch and mouse events, take care when defining separate actions on both mouse and touch events so that you don’t double up. Use event.preventDefault() inside touch event handlers to keep the browser from firing the corresponding mouse events, too. (This has some repercussions, which we’ll discuss.) For example, if you want to do something on touchstart for touch and mousedown for mouse, you need to tell the browser not to process any other touch events when touchstart is triggered or it will do the mousedown action too: