Read an Excerpt

Developing Virtual Reality Applications

Morgan Kaufmann

Chapter One

1.1 WHAT IS VIRTUAL REALITY?

When we speak of "virtual reality" (VR) we refer to a computer simulation that creates an image of a world that appears to our senses in much the same way we perceive the real world, or "physical" reality. In order to convince the brain that the synthetic world is authentic, the computer simulation monitors the movements of the participant and adjusts the sensory display or displays in a manner that gives the feeling of being immersed or being present in the simulation. Concisely, virtual reality is a means of letting participants physically engage in some simulated environment that is distinct from their physical reality.

Virtual reality is a medium, a means by which humans can share ideas and experiences. We use the word experience to convey an entire virtual reality participation session. The part of the experience that is "the world" witnessed by the participant and with which they interact is referred to as the virtual world. However, the term "virtual world" does not only refer specifically to virtual reality worlds. It can also be used to refer to the content of other media, such as novels, movies, and other communication conventions.

Here is a more formal definition for virtual reality from Sherman and Craig:

A medium composed of interactive computer simulations that sense the participant's position and actions, providing synthetic feedback to one or more senses, giving the feeling of being immersed or being present in the simulation.

Note that the definition states that a virtual reality experience provides synthetic stimuli to one or more of the user's senses. A typical VR system will substitute at least the visual stimuli, with aural stimuli also frequently provided. A third, less common sense that is included is skin-sensation and force feedback, which is jointly referred to as the haptic (touch) sense. Less frequently used senses include vestibular (balance), olfaction (smell), and gustation (taste).

There are many specialty hardware devices involved in bringing the rendered sensory images to the user from the proper perspective. A familiar VR visual display device is the head-mounted display (HMD). An HMD is a device that the user wears on the head, containing a screen positioned in front of each eye. Another common technology used to display the visual part of a VR experience is to project the images onto a large screen or multiple screens that cover a sizable amount of the participant's view. Such displays date back to flight simulation projection domes and to the work of Myron Krueger (an early VR researcher) in the 1970s. This type of VR visual display is generically referred to as a large-screen stationary display.

As our formal definition suggests, an equally if not more important aspect of a virtual reality system is sensing the participant's position. Without information about the direction the user is looking, reaching, pointing, etc., it is impossible for the VR output displays to appropriately stimulate the senses. Monitoring the user's body movements is called tracking.

There are some related technological terms that are also often used in the discourse of virtual reality technology. However, these terms are not necessarily restricted to VR. One such term is "cyberspace." Cyberspace is the notion that people who are physically located in disparate physical locations can, through the use of some mediating technology, interact as if they were physically proximate. Thus, even technology such as the telephone can put two or more people in the same cyberspace.

Two other terms related to virtual reality and to one another are "telepresence" and "augmented reality" (AR). Telepresence is similar to VR, in that it is a means to virtually place a participant in another location in which they are not physically present. The difference from VR is that this location is actually a real place that for one reason or another is too difficult, dangerous or inconvenient for the person to visit in person. Like telepresence, augmented reality gives the user an altered view of the real world. However, the view they are given is of their current physical location, and using technology with many characteristics in common with virtual reality, additional (virtual) information is added to their normal sensory input. Frequently, it is the visual sense that is augmented, providing the user with abilities such as peering through walls, or into a patient's body.

1.2 THE BEGINNINGS OF VR

If one considers virtual reality to be the simulation of an environment that allows a person to experience some place and event other than where they actually are and what is actually happening around them, then flight simulators are an early example of this medium. Flight simulators based on interactive computer displays date back to the early 1970s. Earlier flight simulators made use of mechanically driven instrument displays driven by linkages to the pilot's flight controls such as the yoke, rudder pedals, etc. Many of the precomputer flight simulators were pedantic mechanical devices to give a future pilot the opportunity to become familiar with the flight controls and displays.

Later, by controlling the motion of a video camera over a scale model of some terrain, a sense of immersion was created. Although this did fulfill the criteria for virtual reality portrayed in the opening paragraph of this section, these early flight simulators were not general-purpose environments. A different simulator must be constructed for each type of aircraft, and additional terrain models created for new locations. General-purpose simulation was only possible after the advent of advanced computer graphics and display technologies.

In the following 11 examples of research efforts of different groups in VR development one can gain a sense of how VR technology came to be.

1.2.1 Morton Heilig's Sensorama

Early sensory display experiences included the Sensorama. The Sensorama was the brainchild of cinematographer and inventor Morton Heilig. Demonstrated in 1956, Sensorama was a scripted multimodal experience in which a participant was seated in front of a display screen equipped with a variety of sensory stimulators. These stimulator displays included sound, wind, smell, and vibration. The noninteractive scenario was driving a motorcycle through an environment with the appropriate stimulators triggered at the appropriate time. For example, riding near a bus exposed the rider to a whiff of exhaust.

The Sensorama system, however, was lacking a major component of the modern virtual reality system: response based on user's actions.

1.2.2 Ivan Sutherland's vision for computer-based virtual reality

In 1963, Harvard graduate student Ivan Sutherland demonstrated Sketchpad, a system to allow interactive, computer-generated visual imagery displayed on a cathode ray tube. In 1965, he described a vision for an immersive, computer-based, synthetic-world display system. His vision included the presentation of visual, aural, and haptic feedback in appropriate response to the user's actions. By 1968, Sutherland (as a professor at the University of Utah) had realized and publicly demonstrated a system that accomplished the visual component of his vision.

Sutherland's system included an HMD, mechanical head tracking using spooled retractable cables, and a computer program that rendered a simple stick representation of a cyclo-hexane molecule in three dimensions.

Sutherland later cofounded Evans and Sutherland Computer Corporation (E&S) and developed sophisticated real-time graphics rendering hardware for the flight simulator community.

1.2.3 Myron Krueger's Videoplace

Following Sutherland's demonstration, a variety of research and development efforts were born in university laboratories, government and military facilities, and, later, in the commercial sector.

In the academic community, University of Wisconsin researcher Myron Krueger was experimenting with a different perspective on virtual reality systems, which he referred to as "Artificial Reality." Whereas Sutherland's head-mounted display was especially suited for a first-person point of view in the virtual world, Krueger's artificial reality provided a second-person view of a virtual world in which participants could watch themselves within the world.

Krueger's systems also differed from Sutherland's work in that he used video camera inputs to track the user's movements. Use of video camera technology resulted in two significant differences: The machine's perspective of the user was from the second-person point of view, and the user was not encumbered by any mechanical devices or other sensors attached to their body.

Other universities pursued various aspects of the virtual reality problem.

1.2.4 University of North Carolina at Chapel Hill

In the late 1960s, the University of North Carolina at Chapel Hill (UNC) computer science department founder and professor Fred Brooks espoused the need to have development work geared toward specific application problems. For example, a chemist would be interested in how two molecules dock together. Brooks' team also measured the benefits and pitfalls of their various innovations.

Due to the unavailability of capable hardware at the time, UNC also had to focus on hardware development, including high-performance graphics engines, head-mounted displays, and a variety of input and output devices, including devices to provide haptic feedback in the form of responsive forces. Several commercial products have evolved from the innovative research at UNC.

1.2.5 Electronic Visualization Lab at the University of Illinois at Chicago

At the University of Illinois at Chicago, Tom DeFanti and Dan Sandin cofounded the Electronic Visualization Lab (EVL), where different types of graphical representations, input and output devices, and interaction techniques were explored. Most notable among their achievements were the development of the Sayer glove in 1977 (a glove outfitted to sense the bend of the wearer's fingers) and, in 1992, the announcement of the CAVE™ visual display system. The CAVE is a walk-in virtual reality theater typically configured as a 10-foot cube with three or more of its surfaces rear-projected with stereoscopic, head-tracked, computer graphics.

1.3 VR PARADIGMS

While we have already mentioned that VR systems provide synthetic stimuli to the senses, it is important to note that there are multiple ways by which this can be accomplished. Many suitable display technologies exist, but in general they can be categorized into three display paradigms. These three basic paradigms hold for not only visual displays, but also for display to other senses such as aural and touch (haptic) display systems. Stationary displays are fixed in place. Although the display doesn't move, the world is rendered in response to the user's bodily position. Examples of stationary visual displays include CAVE-type systems, single large screen systems, and desktop monitors. Loudspeakers are an example of stationary aural displays.

Head-based displays move in conjunction with the user's head. Consequently, no matter which way users turn their head, the displays move, remaining in a fixed position relative to the body's sensory inputs. Thus, visual screens remain in front of the users' eyes, and headphones on their ears. Examples of head-based visual displays include the helmet-type display often seen in popular media, and BOOM™-type displays which are a display box into which a user peers that can be moved around on mechanical linkages. Headphones are an example of head-based aural displays.

Hand-based displays are a special case of the head-based paradigm. In this case, users hold the display in their hand. For visual hand-based displays, monitoring both the user's head position as well as the position of the display is required, because the direction of view is important. Most often visual hand-based displays are used to overlay computer graphics imagery registered with the real world. An example of a haptic hand-based display is the SensAble Technologies PHANToMTM arm. The PHANToM provides a dual role by mechanically tracking the user's hand as well as providing a force display to the hand.

1.4 COLLABORATION

One of the strengths of virtual reality is its capability to transcend the barriers of time and space. This transcendence leads to VR being a good vehicle for supporting collaboration. VR environments can foster collaboration in a number of different ways. Space can be shared, either physically or virtually. Dialog can be held synchronously, or in an asynchronous form.

Large-screen stationary systems such as the CAVE are the best type of VR system for collaborating in the same physical space. Many participants have a concurrent view of the virtual world, allowing them to point out items of interest to one another.

Most forms of VR systems provide a good way to collaborate in the same virtual space.

A major benefit of virtual shared spaces is that they allow collaboration to take place via computer networks. Thus, not only can two workers share a space while remaining in their offices just down the hallway from one another, but they can also be an ocean away.

(Continues...)

---

Excerpted from Developing Virtual Reality Applications by Alan B. Craig William R. Sherman Jeffrey D. Will Copyright © 2009 by Elsevier Inc. . Excerpted by permission of Morgan Kaufmann. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---