AT&T Laboratories
Cambridge
|
|||||
|
|
AT&T Laboratories |
|
|
|
|
|||||
Previous Augmented Reality (AR) systems inside buildings have been tethered or restricted to small volumes of space. We believe it is important to deploy throughout a large and populous area in order to examine the potential of mobile AR. Therefore we have chosen to allow the AR user to roam freely within an entire building. At AT&T Laboratories Cambridge we provide personnel with AR services using data from an ultrasonic tracking system, called the Bat system, which has been installed building-wide. The deployment process highlights practical issues such as cost and ease of installation. Furthermore, if large numbers of people are working in the augmented environment on a day-to-day basis, we are forced to consider the social integration aspects of the system.
We have approached the challenge of implementing a wide-area, in-building AR system in two different ways. The first uses a head-mounted display connected to a laptop, which combines sparse position measurements from the Bat system with more frequent rotational information from an inertial tracker to render annotations and virtual objects that relate to or coexist with the real world. The second uses a PDA to provide a convenient portal with which the user can quickly view the augmented world. These systems can be used to annotate the world in a more-or-less seamless way, allowing a richer interaction with both real and virtual objects.
A software architecture for Sentient Computing has also been implemented at AT&T, using the bats and other sensors to update a model of the real world.
The bat system is installed throughout our three floor, 100,000 cubic foot office, which has over 50 rooms. The system is continually used by all 50 staff, and tracks over 200 Bats. The Bats have a battery lifetime of 12 months. The ultrasonic receivers are mounted recessed in the centre of the ceiling tiles, with cables in the roof, which makes the tracking infrastructure extremely unobtrusive.
The world model currently contains 1900 software objects corresponding to personnel, telephones, computers, walls, windows, etc. in the real world. In this project, we aim to utilise the detailed data set inherent in the sentient system to provide users with a rich AR experience.
The HMD software object takes the current estimate of head orientation and uses a non-linear filter to make a new estimate, based on the latest raw measurement. The effect of the filtering is to apply very small, heavily damped corrections to the estimate of orientation when head motion is slow. When head motion is faster the corrections are much larger, and when movement is very rapid the next estimate of orientation is immediately taken to be true. This filter is implemented using the technique of spherical linear interpolation (SLERP).
The inertial tracker only provides orientation updates, but it is less crucial to provide frequent estimates of head position than orientation, as angular velocities result in much larger image velocities than those caused by translational velocities. The estimate of orientation provided by the inertial tracker is prone to drift, so we use the Bats to correct for this in the medium-to-long-term, and rely on the inertial tracker in the short periods of time between Bat readings.
Each time an estimate of the head orientation is made by the HMD software object the estimate is communicated using CORBA to a process running on the laptop. This estimate is compared with the most recent reading from the inertial tracker, and a filtered correction to the inertial tracker is then calculated.
A series of calibration screens ensure the user is wearing the HMD properly, and guides them through the procedure. The user clicks their Bat over a cross-hair in the centre of their field of view. The user is requested to keep their head level (no roll) so that both a view direction vector and up-vector can be determined.
The figure below shows a typical view through the HMD. In this case the system has labelled a user, a computer (hostname tamarillo) and a telephone (number 498). As the person (or other object whose position is monitored by the sentient system) moves, the label follows them in the user's view.
 |
|
|
The Batportal is a lightweight alternative to the HMD using a hand-held PDA. This benefits portability and ease of use. The display is, of course, non-immersive and the tracking capabilities are less effective than the HMD system.
The Batportal's screen differs from the HMD in two significant respects: firstly it is not transparent, and secondly the viewing frustum is very narrow, particularly at arm's length. The lack of transparency is not a problem because the device is small, so the user can easily see the real world context around the screen. To overcome the narrow viewing angle we use a false perspective, giving a ``fish-eye lens'' effect. Consequently, registration does not have to be precise, since objects are not seen as directly overlaid.
The viewing angle can be adjusted using the iPAQ's cursor keys. We have also implemented a mode in which the ``magnification'' can be continuously adjusted by holding the Batportal closer or further away from the eye.
Speech is currently used to identify which room the user is in, and to provide feedback when the current user or mode changes. Status information that can be communicated aurally does not clutter the limited screen area, and makes the device friendlier in use.
Sound output could be a distraction in a busy office environment, so for audio-intensive applications we use a lightweight single-sided earphone. For example it should be possible for the Batportal to narrate the subject lines of e-mails or answer queries about the environment even when the user is walking down the corridor or sitting in a meeting.
Our sentient computing system provides a model of the world which includes objects such as computers, furniture, phones and personnel. Not only do we know the physical locations of these objects, but we have access to other properties and state, such as which people are visitors and whether a phone is on or off the hook. We can use the AR system to augment the user's view of these objects with annotated labels. Colour coding is used give cues to the state of the object they are labelling.
Annotation can be applied to fixed points in space (such as a room) or to moving tagged targets such as other people. This is a useful way of checking someone's name, office and perhaps common interests and so on. The sentient computing platform allows this information to be shared by all users of the AR system, whether they are using the HMD, Batportal or a traditional interface on a PC.
An interesting class of AR application involves navigation within buildings - these include finding one's way around, locating another person or following a personal augmented tour of the building.
We can use our AR systems, together with our sentient computing environment, to display the locations of people, walls, computers, telephones and other objects relative to the user. The level of augmentation can be varied to support the particular task that the user wishes to achieve. For example, suppose the Batportal system renders a 3D view of the current state of the building. Walls can be switched between opaque and transparent, giving the device an ``X-ray vision'' capability. The user can also choose to display the structure of the entire building or just the current floor.
Navigation is possible using various means such as virtual signposts, a 2D map, compass arrows or turning signals. Virtual marker objects can be created by pressing a trigger button on the user's Bat (in the HMD system) or iPAQ (in the Batportal system), or by utilising a mode in which a virtual marker is automatically placed every half-second to create a trail, showing the route taken through the building by the user.
Normally, these active points in space (known as virtual buttons) are physically labelled by a post-it note or poster. However, we can extend this interface within the personal space of the user of the HMD system by dispensing with the physical labels, and relying on the AR annotation of the physical point to indicate that a virtual button is present, and what that virtual button controls. This approach has the advantage of reducing the amount of visual clutter in the environment, and has proved to be practicable.
For example, in a supermarket a simple 2D map on a Batportal could assist with locating items and indicating routes, as well as highlighting special offers and items which have been purchased before. The screen could also be used to display prices, ingredients and recipes.
Consider a hypothetical museum example. Museums are attractive environments for AR systems, because the infrastructure only has to be installed once, after which it is unnecessary to physically label objects when exhibitions change. Meta data can be added directly to the virtual world in a way which is complementary to, but easier than, creating physical signs or guidebooks.
The AR systems described here are personal devices, and so do not interfere with other visitors' experiences, and can be customised to take account of each user's age, language, interests and preferences. For example, one could request that the history of each painting be displayed on approach, the titles of modern art be withheld and any African sculpture nearby be highlighted. Furthermore a personal guided tour could be created with a different emphasis from the standard order of presentation.
The system could behave quite differently for children on a school visit than for ordinary visitors. Functions would include drawing attention to objects or aspects which the teacher considers important, or monitoring a ``treasure hunt'' for particular items (say three pictures which contain a mermaid, on discovery of which the students are rewarded with pop-up information to complete a worksheet). The teacher can readily check if the objectives have been completed, and co-operation is also possible, since routes to interesting places can be transmitted to peer AR systems.
 |
 |
|
| David Ingram | Joseph Newman |
This paper appeared in Proceedings of the 2nd IEEE and ACM International Symposium on Augmented Reality (ISAR 2001), October 2001, New York.
For comments, suggestions and further information please contact us.
Copyright © 2001 AT&T Laboratories Cambridge