Personal computing is in an awkward adolescence right now. On one hand, we are rapidly moving into ubiquitous computing environments that let people constantly interact with the omnipresent network; on the other, the devices and interfaces we are using to enter these new frontiers provide woefully inadequate user experiences. Let’s take a look at one of the key technologies that will take mobile user experiences to the next level: holography.
Holography and the State of Input
The primary reason why the BlackBerry® became such an enormous success is its miniature QWERTY keyboard, which lets people rapidly enter information and, in the process, made easy-email-while-on-the-run a reality. Earlier devices such as cell phones and Palm® PDAs provided a substandard means of communicating with a computing system, but the BlackBerry took the well-established and long-practiced QWERTY keyboard interface and employed it in a practical and portable form. This allowed people to engage in a more natural human/computer interaction.
Champion Advertisement
Continue Reading…
Unfortunately, while it is our accustomed way of entering information into a computing system, even in its most optimal state, using the QWERTY keyboard is an unnatural and awkward activity. Shrinking it down to fit on a handheld device may have allowed mobile computing to leap far forward, but this remains a poor and ultimately temporary solution to the problem of mobile data input. While more people—particularly people in east Asian cultures—are embracing an active mobile-computing lifestyle, the modes of input mobile devices offer have not advanced much. Components that behave like a mouse and miniature touch screens cannot replicate the speed and power of desktop computing environments.
The potential of mobile computing is enormous. Indeed, despite clumsy input devices, the mobile computing market is taking off and will only continue to get bigger—even if input devices do not get demonstrably better. Yet, unlocking a superior user experience for mobile computing begins with the development of more sophisticated modes of input. Soon, more effective voice recognition will drive input, in part. And, eventually, modes of input will evolve into what we have come to think of as science fiction—controlled by our eyes, our gestures and body movements, and ultimately, perhaps the holy grail of direct mind-to-machine interaction. But in the space between, we need a better, yet technologically practical way of entering data.
Most people who use computers know and are accustomed to using the QWERTY keyboard. For all of its faults, it remains both the primary and the most rapid, yet robust way for people to communicate with computers. That is why incorporating it into BlackBerry devices was such a success. However, miniaturization and thumb typing compound the issues of a full-size keyboard. There are already reports of heavy usage of the BlackBerry causing physical problems in users’ hands and wrists—the new repetitive stress injury BlackBerry thumb.
The obvious opportunity might seem to be a mobile device with a larger QWERTY keyboard. But others have done this before. Palm has long offered a fold-up, peripheral keyboard that somewhat conveniently fits into a coat pocket. But it is just one more thing to carry, among an ever-increasing cache of personal gadgets, and ultimately, lags in performance compared to a standard, full-size keyboard. At the same time, notebook computers might seem to be effective mobile devices—most sporting a full-size QWERTY keyboard, along with full desktop-computing power. Unfortunately, notebooks are just too big and bulky to function as truly mobile devices. While portable enough to carry around, they remain a far cry from a cell phone or PDA-sized device that a user can slip into a pocket, without being noticeably encumbered in any environment—short of a nudist colony.
So there is a tension between our desires for usability and mobility: the QWERTY keyboard is an effective and well-practiced input device, but is too large and bulky for use in a truly mobile user experience. Enter the holographic keyboard—virtual save for a small component that is part of a mobile computing device. A holographic projector could place a full-size keyboard anywhere, giving us a computer input experience that is similarly robust to that of our desktop/place-bound computing stations. A holographic keyboard synthesizes the essential usability problem of providing desktop-quality computing with seamless and always-available portability. Now, I’m sure the early versions of such a technology will not deliver on the promise of the idea, but this is the likely technology to take us from the awkward fumbling of today into the powerful mobile computing environment of tomorrow.
Holography and the State of Output
But input is only part of the problem. One of the biggest challenges facing everyone in the mobile space today—on both the hardware and software sides—are the device outputs, particularly their displays. The screens are incredibly small. The resolution is poor. The output of even the most remarkable devices of today pales in comparison to the standard 14-inch desktop monitors of some 15 years ago—a monitor that today is essentially obsolete. Indeed, for desktop computing environments, we are rapidly moving into the world of large LCD flat-screen displays—some in excess of 30 inches. Why bigger? Because when it comes to visual output, bigger is almost always better. More pixels. Higher definition. Improved usability and a more natural interactive environment. This is what makes the “Baby Faces” of mobile devices—as Aaron Marcus so cutely and appropriately calls them—so inadequate for displaying computer output.
The primary constraint is again one of portability. Notebook computers offer relatively effective visual output, but as I mentioned earlier, are not truly mobile devices. They do not integrate well into our on-the-move lifestyles. So, how can we provide a large, high-resolution display that is truly portable—small enough to fit in a pocket?
Nothing even remotely close to such a technology exists today, but as with the virtual QWERTY keyboard, we could once again use holographic technology to create a projection of a computer display or even a virtual 3D environment. Holography would enable us to project a large—even full-size—virtual object with next to no physical space requirements. It might take some time to develop very high-resolution holographic displays, enabling this technology to achieve display quality that is comparable to that of a desktop monitor. However, the sheer size of such a display would, to some degree, make up for any deficiencies in resolution. Just in their being much larger than the mobile output devices that are now standard, holographic displays would exceed the current mobile user experience—to say nothing of their potential for three dimensionality, which would open up entire new frontiers of interacting with our machines.
Privacy Concerns, and How to Solve Them
Implicit in next-generation interface technologies—holographic output displays as well as rich voice-enabled inputs—are problems relating to privacy. And these are significant problems. For users, the confidential information that is a normal part of living a digital lifestyle—from banking and purchases to password entry and communicating secret personal information for account verification purposes—becomes dangerously public in these new contexts. This is to say nothing of the personal privacy we prefer to have when communicating with a lover or getting dressed-down by our boss or playing a computer game. These more effective means of interaction bring with them very real challenges to our personal security and comfort.
At the same time, there are also privacy concerns for non-users. Already today, people complain bitterly about others talking on cell phones in public spaces. What if voice inputs were extended beyond voice communication and became the driver for most human/computer interaction? What is now a relative annoyance would become an unbearable cacophony. Then consider the physical space requirements of holography: even though these are virtual objects, the reality is they fill space just as real objects do. Others would need to walk around a projected computer display just as they would need to walk around a physical one. The problem will amplify if and as more and more people adopt and use the technology. The next time you go into your local Starbucks® during the morning rush, imagine half or more of the people seated inside with 15-inch computer monitors on tables a few feet in front of them. What impact would that have on the experience? How would that change the very personality of Starbucks? After all, if these technologies work, people will use them. The relatively modest number of notebook users today would undoubtedly be replaced by an ever-increasing number of rich-mobile-device users.
As is often the case, technology will need to solve the problems that technology creates. I saw a great example of a possible technology solution in practice recently, in a government R&D environment: a device that looked something like a DirecTV®-style satellite dish, focused and directed music in a specific and completely unexpected way. We started in one room, where music was very lightly playing on iPod® speakers. Then we went outside the room and walked down a really, really, really long hallway. Our colleague took the device, stood outside the office, and tilted it in a way to create triangulation. Suddenly, we were able to hear the music at approximately the same volume level as we had while in the office! If we moved a few feet to the right or left, we couldn’t hear it at all. But if we stood within the correct field, we were able to hear it. No one else outside the room in this busy work environment was able to hear it at all. It was really an amazing experience.
Whether or not that technology in particular will ultimately be the one to solve some of the privacy problems next-generation interfaces create, it underscores the essential truth that there is already technology out there that is well ahead of anything we are seeing in mainstream use today. Historically, technology has been successful in solving the problems it creates. With every gain come inevitable challenges. If we are to enjoy the power and freedom of robust mobile user interfaces, we will need to develop technologies that can help keep our computing experiences somewhat contained and, therefore, private.
Filling the Space Between
Yes, the more distant future will be one of entirely digital environments, in which inputs and outputs are all around us, and it is only a device containing our personal data—or a key into our personal data—that serves as our mobile computing device. But in most parts of the world, we are a decade or more away from realizing that degree of integration. In the meantime, we are dealing with ubiquitous computing experiences that are painfully low resolution in every way, presenting a real opportunity for a technology or company that can change this very tepid paradigm. Holography is one technology that very nimbly addresses issues of both input and output. Whether applications of the technology will reach the point where they are both sufficiently effective and affordable to the mass market—as well as relatively private and secure—remains to be seen. But this is certain: the trend that digital computing technologies have started—transforming elements of the physical world into a virtual existence—is one that will continue in the decades ahead. Holographic technology is an early technology that could serve an important role in not only the short-term improvement of the mobile user experience, but also in changing our very mental model of the relationship between the real and the virtual.
Good article Dirk. We are rapidly seeking new innovative ways to interact with technology. The iPod interface is a superb example of the new breed. Simple, effective and easily learnt by anyone. A circle cannot be complicated!
You’ve solved it! Make sure to include a holographic keyboard with motion sensing devices to register keystrokes. To counter privacy concerns, create cloaking technology, perhaps using an electronic version of polarized optics. This raises some questions: “Would people ever leave their virtual pods? And what will it be like to move through 3D space?
The most promising implementation idea I’ve ever heard is Augmented Reality … Virtual reality on top of the real world.
Technically, you’d wear glasses that can overlay virtual images on the real world. They’d track your position and head orientation, so that virtual objects could be connected to real ones. This would also neatly solve the privacy issue.
A colleague and I can sit at a conference table, shuffling papers and folders, and a casual passer-by just sees two people with glasses, shuffling air.
I think you jinxed yourself with the words “Nothing even remotely close to such a technology exists today”. This was not actually true when the piece was written, although the fact that it was an error wasn’t public knowledge. All the more impressive you targeted holographics as the enabling technology that would drive the next generation.
Presented for your consideration:
INPUT - http://www.thinkgeek.com/computing/input/8193/
Bluetooth Laser Virtual Keyboard. This tiny device laser-projects a keyboard on any flat surface. You can then type away accompanied by simulated key click sounds.
Light Blue Optics has developed a unique,
patented laser projection technology that
exploits the principles of diffraction to project full-color, real-time video images.
Our projection technology uses laser light sources in conjunction with a phase modulating liquid crystal on silicon (LCOS) microdisplay on which a diffraction pattern, calculated from the desired image, is displayed. When the microdisplay is illuminated by coherent laser light, the desired image is projected.
As a social futurist, Dirk envisions solutions to system-level problems at the intersection of humanity, technology, and society. He is currently the managing director of SciStories LLC, a design agency working with biotech startups and research scientists. In addition to leading SciStories, Dirk is a co-owner of Genius Games LLC, a publisher of science and history games. He also cohosts and produces Creative Next, a podcast and research project exploring the future of creative work. Dirk has been a design entrepreneur for over 15 years, has raised institutional venture funding, and has enjoyed two successful exits. He earned a Master of Arts from the prestigious Popular Culture program at Bowling Green. Read More
