The rising field of wearable computing brings with it a unique set of requirements as well as unique possibilities for visual interfaces. This paper explores these requirements and possibilities, focusing on the "Eyetap" system for "Mediated and Diminished Reality" as implemented and described by Prof. Steve Mann. The theoretical underpinnings of the system and the engineering decisions driving its implementation will be discussed as well as the current and potential applications.
"A wearable computer is a computer that is subsumed into the personal space of the user, controlled by the user, and has both operational and interactional constancy, i.e. is always on and always accessible. Most notably, it is a device that is always with the user, and into which the user can always enter commands and execute a set of such entered commands, and in which the user can do so while walking around or doing other activities. The most salient aspect of computers, in general, (whether wearable or not) is their reconfigurability and their generality, e.g. that their function can be made to vary widely, depending on the instructions provided for program execution. With the wearable computer (WearComp), this is no exception, e.g. the wearable computer is more than just a wristwatch or regular eyeglasses: it has the full functionality of a computer system but in addition to being a fully featured computer, it is also inextricably intertwined with the wearer. This is what sets the wearable computer apart from other wearable devices such as wristwatches, regular eyeglasses, wearable radios, etc.. Unlike these other wearable devices that are not programmable (reconfigurable), the wearable computer is as reconfigurable as the familiar desktop or mainframe computer."from: Definition of "Wearable Computer" (http://wearables.about.com/gadgets/wearables/bldefinition.htm)
An Eyetap is a system for interposing a computer into the wearer's field of vision allowing it to mediate what the wearer sees. Visual information can be blocked, modified, or added in real-time. Typically, an Eyetap system resembles an ordinary pair of eyeglasses or sunglasses and consists of three parts:
Several different implementations of the Eyetap have been implemented exploring different focus models. In one kind of implementation, a focus tracking mechanism is used to reconstruct rays of diverted light in the same depth plane as imaged by the camera. In another implemenation, the aremac has an unlimited or extended depth of focus so that the eye itself can focus on different objects.
"Mediated/Augmented Reality is the ability of the computer to offer enhanced presentations of reality to the user."from: http://wearables.about.com/gadgets/wearables/library/weekly/aa060100a.htm
Mediated Reality is created when virtual, or computer generated, information is mixed with what the user would otherwise normally see. The Eyetap system, along with camera-based head-tracking is used to absorb, quantify, and resynthesize under computer control, the light that the user would normally see, with information added or removed by the computer. Mediated Reality differs from "virtual reality" in the sense that Mediated Reality allows the visual perception of reality to be augmented, or, more generally computationally altered whereas "virtual reality" typically creates a wholly artificial environment for the user.
Whenever the subject of wearable computers or Mediated Reality is brought up, a typical complaint is that it would merely increase the degree of "information overload" in the lives of its users. Since the human brain can only make sense of so much information at once, this is a legitimate concern. Diminished Reality is a reaction to this fear. It is an application of Mediated Reality that primarily involves subtracting information from the user's field of vision; typically removing "noise" in the form of unnecessary or distracting information. As will be described in a later section, one application of Diminished Reality is the identification and blocking of Billboard type 2-dimensional advertisements, blotting them out or reusing the visual space for more useful information.
In order for the Eyetap system and Mediated/Diminished Reality to be implemented, several problems had to be solved. In order for the Eyetap system to function in ordinary environments, it is necessary that it be designed such that it doesn't rely on any special apparatus being present in the environment. Therefore, a camera-based head-tracking method was developed based on the VideoOrbits algorithm which performs head-tracking, visually, based on a natural environment, and works without the need for object recognition, relying instead on algebraic projective geometry. A primary focus of this paper will be the exploration and analysis of these direct featureless mathematical methods for estimation of a projective coordinate transformation.
Some attention will also be paid to the theory behind several applications of Mediated Reality such as the blocking of 2-D advertisements and the wearable face-tracking "virtual nametag" feature. The former requires sophisticated plane-tracking techniques while the latter utilizes traditional Fourier analysis based techniques for facial recognition and identification.
What makes the field of wearable computing so exciting is the myriad of potential applications. In this paper, many of these applications will be mentioned, but, in the interest of brevity, only a few will be discussed in great detail. For example, one of the more unusual and interesting applications of Mediated Reality is the concept of Diminished Reality and specifically, the ability to identify and block 2-Dimensional advertisements and other unwanted distractions. Useless, annoying and potentially dangerously distracting visual "noise" such as advertisements can be identified based on their general shape as 2-dimensional planes with the help of algebraic projective geometry and distinguished from other desirable planes using more sophisticated pattern recognition techiques (typically hidden markov models or neural networks). The visual real-estate that these advertisements take up can either be blocked out -- replaced with a featureless, solid plane -- or reused for the communication of more important information.
Using traditional methods for facial recognition and identification, a "virtual nametag" system can be implemented that builds up a database of faces of people the user has come into contact with while wearing the device and information relating to that face such as their name, title, and where and when they were previously encountered. When this person is encountered at a later date, the Mediated Reality device, upon recognizing their face, can superimpose the relevant information over the user's field of vision allowing them to essentially never forget a face.
In a similar vien, feature extraction from a landscape can be used to provide superimposed navigation clues and annotations. Eg, while walking down the street, the system may be able to identify the user's location and orientation and superimpose directions to the user's destination or display a reminder over the grocery store on the left that the user should stop and pick up some milk.
Some resources i've discovered in my preliminary investigation of this topic:
Eyetap System (http://wearables.about.com/library/weekly/aa012201a.htm)