Here’s an update of glossary I wrote at least ten years ago, when I was working in VR more head-on. I think this can be useful for the next set of Web 3D discussions, if only to serve as common anchor points for debate. I’m not going to claim that my definitions are the final word on anything, and they’re very basic from an academic point of view. But I think they can help structure the dialog in a constructive way.
So consider the following a basic glossary (with cheap visuals) of virtual worlds:
Conceptually speaking, Virtual Worlds are a tool that can aid our understanding of how information is communicated from one domain to another, ideally when people are involved. They can also help us improve communication, especially when a computer is involved.
In the case of ordinary communication (say, person to person), we can use the idea of virtual worlds to describe how each person models some information in their mind. In the case of two people talking about a car, each person may model things quite differently: more or less symbolically, visually, audibly, concretely, abstractly, objectively, subjectively, or any combination thereof. But these are all thoughts, which are, as of today’s technology, isolated in our minds.
So a communicator must translate those thoughts, that mental model, to some sharable medium, for example, spoken language. And the recipient must re-encode the content of that medium into his or her own mental world, a new model of the car which may be quite different than the communicator’s original thought.
It sounds incredibly obvious and natural, but the processes involved are actually quite complex. The key is that the shared virtual world (the spoken word “car” and whatever else we add) carries only as much meaning as we give it. And it alters the message in predictable ways. A simple verbal conversation can actually convey very rich information — a single syllable can convey a thousand related concepts. But it must do so through descriptive language, which can change meaning, leading to misunderstandings too.
Virtual Worlds exist naturally in a variety of forms such as books, movies, and web pages. But these are often one-way, or read-only, and there is a disconnect in time between the originator and the recipients. The virtual world can be as captivating as anything else if the author is skilled in predicting the reactions of the audience in advance. But the model is fundamentally the same. The author has some mental model. It is communicated, translated into some medium or media. And then someone else experiences it, translating back into their own mental world, to whatever extent possible. For some books, I’d argue, that mental model can be richer than any movie, though personal experiences vary.
Virtual Reality, as defined, is an attempt to use digital media (visual, audio, tactile) which best “fit” our natural ability to communicate, rather than forcing us to adapt to unnatural media as dictated by technology. It is not always the most efficient, but it is meant to be as natural and unobtrusive as possible.
With Human-Computer Interaction, the computer’s input and output mechanisms become the medium for communication. On flat 2-D screens, this limits communication to 2-D imagery (pseudo-3D at best). With virtual reality technology, the computer can communicate in a way which is more compatible with our natural senses, increasing effectiveness, enjoyment, and productivity.
The computer-supported virtual world uses a tightly coupled “sensation-action” loop to simulate natural human communication. We act, the computer senses, reacts visually and otherwise, and we perecive the response, and so on. If this loop is done correctly, a resonance can be created, and the loop becomes more than just the sum of its parts.
If technology is skillfully employed, we also feel a sense of Presence–that we are in the virtual environment, part of it. This helps bridge the gap between the computational world and our mental world. If the virtual world is highly interactive, we may also feel a greater sense of presence. When the computer software is sufficiently sophisticated to provide a high degree of reflectivity, the possibilities of communication expand. We will be able to more fully and effectively express ourselves to the computer and to other people.
There is no strict requirement that any computer-mediated virtual worlds be 3D, but given the definitions of Immersion and Presence below, 3D-ness may help increase those desirable qualities significantly.
These two concepts are closely related, but subtly different. In one sense, you can think of the difference as cause and result. When we want to create a feeling of presence, we may employ immersion as a technique.
But in other ways, it is not such a strict cause-effect relationship. We can be immersed in a good book, or a movie on an old 12″ TV and be completely distracted by a cell phone inside the best VR experience in the world. But in general, the feeling of presence is enhanced by providing contextual cues, peripheral vision, ambient sounds, lighting, and so on. The more of those we provide (in a reasonable fashion) to the human sensorium, the easier it is to create that feeling of presence.
And with that feeling of presence, a person is more cognitively engaged. And, as a result, he or she may more effectively receive, and perhaps communicate, information. Very subtle cues can convey a great deal of information, simply because our brains are more engaged in their detection and processing. Some cues, like smell, go straight to the oldest (from an evolutionary perspective) and most powerful (from a communications perspective) regions of our brains.
Comfort, satisfaction, performance, understanding, and even creativity may all be enhanced with increased presence. The task of understanding and enhancing this feeling is one of the principal challenges to using 3D graphics for communication.
Without interactivity, games would seem like sporting events or movies. We could watch and cheer. But we can have no real impact on the events and on what is ultimately communicated back to us. But it’s a bit more complicated than that. A good movie or sportscast does actually engage us in some way and cause our thoughts (at least) to interact, one-way, with what we experience. For example, a murder mystersy TV show may cause us to suspect one person over another. The writers are counting on that to make the next reversal have a more dramatic effect. So we are interacting, even with a movie. But our interactions are all internal and have no effect on the movie itself.
Reflectivity, then, measures how much our interactions and expectations can change the virtual world that is communicated to us. If a movie reflected our intentions, we would have some control over how it unfolds. Games tend to be more reflective. But even today, the degree to which we can “go off the path” or modify the environment is limited in many games. We can’t always break a chair, even though we think we should. We can’t always ask a question of a non-player-character, though we might want to. So the more reflective something is, the more open-ended it is, and the more we feel we have control over how we (or our character) act and interact.
Resonance is the desired end result. The more interactivity and reflectivity we experience, the more likely we are to react and enhance the process. In the cycle of communication, resonance can actually feed back and cause more presence, more interactivity, more reflectivity.
For example, say we are in a 3D environment and we do try to break that chair. The chair shatters, sending splinters in the air. We reflexively duck to avoid the splinters. That’s a resonant effect, because we chose to break the chair, without necessarily thinking about the next step. And the result is we may feel more present in the environment. If one of those splinters hits us and we are aware of the sensation, that’s another resonant effect, which may make us feel even more connected. We’ve gone through two effective cycles of sensation, action and reaction from one enabling event.
The key is to take that process and apply it to visualizing and interacting with information.