Everyware in the Near Term
Vernor Vinge’s entertaining short story “Synthetic Serendipity” [1]—first published in the IEEE magazine Spectrum in 2004—presents a near-term future that is fully permeated by everyware. The narrative is from the perspective of a high-school student named Mike, who likes to play games.
Though publication of the story in Spectrum does not constitute an official endorsement of the future “Synthetic Serendipity” presents, the story’s appearance there—in combination with Vinge’s reputation, professional background, and outlook—tends to validate its premise as a possible future.
Vernor Vinge
is an award-winning science fiction author and former professor of computer science who anticipated the rise of cyberspace, describing it in his 1981 story “True Names.” Vinge is also the leading exponent for the concept of technological singularity, that moment in the future at which machines eclipse human intelligence, accelerating their own evolution. Spectrum is “the flagship publication of the IEEE, the world’s largest professional technology association.” [2] While Spectrum publishes fiction on occasion, its primary audience is the professional community of engineering and business technology.
“Synthetic Serendipity” begins, “Years ago, games and movies were for indoors, for couch potatoes and kids with overtrained trigger fingers. Now they were on the outside. They were the world.” [1] With this opening, Vinge deliberately inverts two essential aspects of the world most people take for granted—that games are distinct from reality, and media is separate from the wider environment—thereby requiring readers to suspend their mental models of play and entertainment.
The remainder of “Synthetic Serendipity” elaborates on this inversion, showing a world in which alternate-reality games (ARG’s) and augmented reality (AR) have converged with ubiquitous computing to erase the boundaries between entertainment and real life.
As the story progresses, protagonist Mike Villas is playing a game with friends while walking to high school. Vinge describes the augmented landscape around the group of boys:
“The canyon was not a dead zone. Not at all. County Flood Control kept the whole area improved, and the public layer was just as fine as on city streets. As they walked along, Mike gave a shrug and a twitch just so. That was enough cue for his Epiphany wearable. Its overlay imaging shifted into classic manga / anime: the manzanita branches morphed into scaly tentacles. Now the houses that edged the canyon were heavily timbered, with pennants flying. High ahead was a castle, the home of Grand Duke Hwa Feen in fact, the local kid who did the most to maintain this belief circle. Mike tricked out the twins in manga costume, and spiky hair, and classic big-eyed, small-mouthed features. ‘Hey, Jerry, look.’ Mike radiated, and waited for the twins to slide into consensus with his view. He’d been practicing all week to get these visuals.” [1]
From the design perspective, this paragraph is dense with implications for the nature of the everyware experience. Vinge presents a world in which the average adolescent seemingly has free and ubiquitous access to a pervasive, geo-localized information and communications infrastructure layer the local government maintains. Further, Mike Villas and his friends use this pervasive network to join an active, geographically distributed community that stages complex, extensively customized alternate / augmented-reality games. Mike and the other participants play these augmented reality games using a range of sophisticated wearable computing-hardware devices.
These wearable systems employ gesture, audio, video, haptic feedback, and possibly other sensory channels as their primary interaction and control mechanisms. This is not the world of the keyboard and mouse we know today and goes far beyond multi-touch. Synchronously shared personal displays with stylized visualization themes let game players define and exchange their individual viewpoints on the augmented-reality landscape surrounding them. Broadcasting and sharing players’ individual views facilitates cooperation and competition. Players create, manage, and enjoy games by exchanging their reference points. Shared augmented visualizations, integrated social elements, and communication capabilities such as presence, identity, and conversation let players maintain layered, nuanced dialogues with other players and people who are either nearby or linked remotely from afar.
All of these social and co-creative dimensions allow, or perhaps—given common social pressures for self-presentation and peer sociality—require players to define and manage multiple digital identities. [3] Technically, their avatars—the digital identities Mike and his friends create for playing games—persist outside game contexts, remaining present as elements of the networked public space. [3, 4]
In addition to the pervasive information layer in public areas of the landscape that is devoted to utilitarian purposes, some private-property locations serve as enhanced game environments that offer richer experiences for players, or co-creators. Vinge describes an example of this sort of public / private area as follows:
“Pyramid Hill had all the latest touchy-feely effects. These were not just phantoms painted by your contact lenses on the back of your eyeballs. On Pyramid Hill, there were games where you could kick lizard butt and steal raptor eggs, or games with warm furry creatures that danced playfully around, begging to be picked up and cuddled. If you turned off all the game views, you could see other players wandering through the woods in their own worlds. Somehow the Hill kept them from crashing into each other.” [1]
Some of the identifiable mechanisms of everyware to which Vinge refers include
- three-dimensional projections of information onto contact lenses
- multiple game styles and visualizations
- overlapping networks of players who are either physically present or remotely and digitally present
Vinge describes a system that smoothly coordinates all of these interacting elements throughout an augmented-reality domain.
In the park, Mike and his companions encounter a large dinosaur creature of Mike’s design, which promptly ‘eats’ one of them.
“‘Ahh!’ the monster opened its mouth and vomited horror. It was scary good. Mike flicker-viewed on reality: Fred was standing in the steaming remains of his raptor. His shirt was pulled out of his pants, and he was drenched in slime, real, smelly slime. The kind you paid money for.
“The monster itself was one of the Hill’s largest robots, tricked out as a member of Mike’s new species.” [1]
Here, we can see that Pyramid Hill offers an entertainment environment populated by large, autonomous or remotely managed robots. Its multi-sensory special effects, adaptive system rules, and shared visualizations support the game experience. Altogether, Pyramid Hill sounds a bit like Disney’s trademark imagineering approach to amusement park design—though the enormous socio-technical possibilities of everyware amplify its consequences.
When Mike arrives at school, everyware is also part of his out-of-game experience:
“Indoors, the choice of view was not entirely his own. Mornings, the school administration required that the Fairmont School News appear all over the interior walls…. He followed the little green nav arrows with half his attention…and abruptly realized he had climbed two flights of stairs. School admin had rearranged everything since yesterday. Of course, they had updated his nav arrows, too. It was a good thing he hadn’t been paying attention.” [1]
Vinge’s future school directly controls some aspects of what students’ wearable systems display. The school provides personalized wayfinding, based on both Mike’s course schedule and facilities management decisions. In design terms, Mike’s wearable is coordinating multiple organizational and procedural layers of rules and information to direct him to his next assigned classroom, using time and location awareness—which are presumably supported by a mixture of sensors, mapping / cartography, and geographic coordinate systems. While this may be convenient for students who are thinking of other things on their way to class, the ability to literally control what someone else sees raises critical questions about privacy, surveillance, and control. School policy in “Synthetic Serendipity” requires students to appear in person:
“That was why the school made you show up in person for most classes; you had to pay a little bit of attention just because you were trapped in a real room with a real instructor. Chumlig’s lecture graphics floated in the air above them. She had the class’s attention; there was a minimum of insolent graffiti nibbling at the edges of her imaging.” [1]
Like the dramatic reframings that open the story, this description of school policy represents another profound inversion. In the world of everyware, physical presence in a classroom setting limits students’ range of activities—particularly, their access to social interactions, other information, and modes of communication—rather than enhancing access to knowledge, instruction, and the professional guidance of teachers. Virtual absence constitutes truancy—which merely reflects physical absence.
In addition to affecting administrative policy, everyware permeates educational practices and tools in “Synthetic Serendipity”—such as the three-dimensional display teachers use to focus the class’s attention. And it seems Vinge’s future is at least partially hackable, given the mention of graffiti.
In class, Mike uses many of the same elements of everyware that support the augmented-reality games he plays in his free time, letting his attention wander “…from viewpoint to viewpoint around the room. Some were from students who’d set their viewpoints public. Most were just random cams.” Here Vinge suggests a mixed palette of shared visualizations, originating from the cameras and sensors of fellow students, as well as cameras with unspecified, though presumably non-human origins. Vinge uses the dense deployment of instrumentation and networked sensors throughout both man-made and natural environments that is stock in trade for many depictions of the future of ubiquitous computing.
As students often do, Mike has mastered skills that let him avoid the attention of teachers while in class:
“Like most students, Mike kept lots of stuff saved on his wearable. He could run a search like this very close to his vest. He didn’t route to the outside world except when he could use a site that Chumlig was talking about. She was real good at nailing the mentally truant. But Mike was good at ensemble coding, driving his wearable with little gesture cues and eye-pointer menus. As her gaze passed over him, he nodded brightly and replayed the last few seconds of her talk.” [1]
Interaction designers might note Vinge’s mention of “ensemble coding,” a carefully learned technique for using wearable computers that minimizes any signs one’s attention is not focused on the social roles and interactions that are supposed to be in the foreground—for example, the relationship between teacher and student. Ensemble coding is an advancement on contemporary usage practices that aim to hide or disguise when people are working with computing devices—the familiar crackberry—forbidden in group settings like personal and professional gatherings, classrooms, and business meetings.
Like business people who have learned the art of working during meetings, Vinge shows Mike Villas putting his attention on other goals:
“Mike sent out a couple hundred queries, mainly pounding on certificate authorities. Even if the certs were solid, people and programs often used them in stupid ways. Answers came trickling back. If this weren’t Friends of Privacy chaff, there might be some real clues here. He sent out follow-up queries….” [1]
This passage makes it clear that skilled and savvy students can discretely conduct sophisticated information retrieval exercises against large collections of locally cached data, as well as the wider cloud of the Internet. Mike is using his equipment and advanced searching skills to investigate the background of his fellow students, part of a business arrangement—called an affiliance—that relies on anonymous agents, brokered contracts, identity certificates, electronic payments, and deep reservoirs of publicly accessible, personal information. He finds “lots about Ralston Blount [an adult student] that was public record, even some legacy newsgroup correspondence. That was always good for shaking up your parents and other grown-ups….” Vinge invokes the stereotype of younger generations adapting to new technologies more quickly than their parents and grandparents and also reminds readers that a world of everyware is a world of mediated public information.
Teachers, with a good eye and their own skills at hiding interactions with computing devices, can use the same techniques to monitor and manage students, as Mike discovers here:
“Suddenly a message hung in letters of silent flame all across his vision:
“Chumlig > Villas: You’ve got all day to play games, Mike! If you won’t pay attention here, you can darn well take this course over.
“Villas > Chumlig: Sorry. Sorry!
“Most times, Chumlig just asked embarrassing questions; this was the first time she’d messaged him with a threat.
“And the amazing thing was, she’d done it in a short pause, when everyone else thought she was just reading her notes. Mike eyed her with new respect.” [1]
Vinge shows a classroom that is thoroughly permeated by digital elements such as presence, surveillance, traffic and activity monitoring, messaging, and communication with semi-autonomous query agents. All of these digital elements directly affect the social norms and dynamics of the classroom environment, which is itself repositioned not as an enhanced means of sharing information, but as a venue with increased barriers to the normal flows and interactions of ubiquitous computing.
“Synthetic Serendipity” presents a wide-ranging vision of the experience of interacting with everyware, in terms of
- enabling devices and user interfaces
- social norms and patterns, in settings ranging from education to business, play and cooperation
- supporting public and private infrastructures
- architecture and land-use practices
- local and supra-local forms and mechanisms of government
- economic and political structures
- shifting demographic and generational power balances
The first four topics are naturally of most interest to user experience designers, but the contexts in which the everyware arises and the environments it permeates are also important.
