Web 3D, Part 5: Glossary

Here’s an update of glossary I wrote at least ten years ago, when I was working in VR more head-on. I think this can be useful for the next set of Web 3D discussions, if only to serve as common anchor points for debate. I’m not going to claim that my definitions are the final word on anything, and they’re very basic from an academic point of view. But I think they can help structure the dialog in a constructive way.

So consider the following a basic glossary (with cheap visuals) of virtual worlds:

Communication and Virtual Worlds

Communication is the process of translating information from one domain to one or more domains through some medium. A domain can be the mental world of a person, the conputational domain of a computer, or the pages of a book. Each medium structures, limits, and enhances the information it carries to some degree, and some media are more suitable than others to certain modes of communication.

Virtual World: [virtual: in effect, but not in substance; world: an internal or external domain.]

  1. A mental model created by a person as a representation of some reality (actual or fictional);
  2. An information model maintained by a computer for computation and subsequent communication;
  3. The Shared region where human and computer meet; where their respective worlds coincide.

Other variations:

Virtual Reality or Alternate Reality: an attempt to use digital media (visual, audio, tactile) to portray a computer-mediated virtual world in a way that best fits our natural modes of communication.

Metaverse: a virtual world, typically a fairly literal representation of a place. It may be 2D, 3D, or more abstract. But using the typical examples from science fiction literature (Snow Crash, etc..), it is presented as a big multi-user 3D space with its own geography, its own rules, only loosely based on reality. Second Life is a good example of a Metaverse type virtual world.

Mirror World: a more literal representation of the real world in digital form. It attempts to map (or mirror) real-world structures, like geography, or the stock market, in 2D or 3D form. Google Earth is an example of Mirror World.

Augmented Reality: a hybrid virtual world wherein virtual elements are overlaid on our visual/audio/haptic sense of the real world to augment the information flow. Most typically, video is overlaid over a live image or see-through display and matched to our dynamic point of view.

Conceptually speaking, Virtual Worlds are a tool that can aid our understanding of how information is communicated from one domain to another, ideally when people are involved. They can also help us improve communication, especially when a computer is involved.

In the case of ordinary communication (say, person to person), we can use the idea of virtual worlds to describe how each person models some information in their mind. In the case of two people talking about a car, each person may model things quite differently: more or less symbolically, visually, audibly, concretely, abstractly, objectively, subjectively, or any combination thereof. But these are all thoughts, which are, as of today’s technology, isolated in our minds.

So a communicator must translate those thoughts, that mental model, to some sharable medium, for example, spoken language. And the recipient must re-encode the content of that medium into his or her own mental world, a new model of the car which may be quite different than the communicator’s original thought.

It sounds incredibly obvious and natural, but the processes involved are actually quite complex. The key is that the shared virtual world (the spoken word “car” and whatever else we add) carries only as much meaning as we give it. And it alters the message in predictable ways. A simple verbal conversation can actually convey very rich information — a single syllable can convey a thousand related concepts. But it must do so through descriptive language, which can change meaning, leading to misunderstandings too.

Virtual Worlds exist naturally in a variety of forms such as books, movies, and web pages. But these are often one-way, or read-only, and there is a disconnect in time between the originator and the recipients. The virtual world can be as captivating as anything else if the author is skilled in predicting the reactions of the audience in advance. But the model is fundamentally the same. The author has some mental model. It is communicated, translated into some medium or media. And then someone else experiences it, translating back into their own mental world, to whatever extent possible. For some books, I’d argue, that mental model can be richer than any movie, though personal experiences vary.

Virtual Reality, as defined, is an attempt to use digital media (visual, audio, tactile) which best “fit” our natural ability to communicate, rather than forcing us to adapt to unnatural media as dictated by technology. It is not always the most efficient, but it is meant to be as natural and unobtrusive as possible.

With Human-Computer Interaction, the computer’s input and output mechanisms become the medium for communication. On flat 2-D screens, this limits communication to 2-D imagery (pseudo-3D at best). With virtual reality technology, the computer can communicate in a way which is more compatible with our natural senses, increasing effectiveness, enjoyment, and productivity.

The computer-supported virtual world uses a tightly coupled “sensation-action” loop to simulate natural human communication. We act, the computer senses, reacts visually and otherwise, and we perecive the response, and so on. If this loop is done correctly, a resonance can be created, and the loop becomes more than just the sum of its parts.

If technology is skillfully employed, we also feel a sense of Presence–that we are in the virtual environment, part of it. This helps bridge the gap between the computational world and our mental world. If the virtual world is highly interactive, we may also feel a greater sense of presence. When the computer software is sufficiently sophisticated to provide a high degree of reflectivity, the possibilities of communication expand. We will be able to more fully and effectively express ourselves to the computer and to other people.

There is no strict requirement that any computer-mediated virtual worlds be 3D, but given the definitions of Immersion and Presence below, 3D-ness may help increase those desirable qualities significantly.

Immersion and Presence

Immersion is a measure of the degree to which information surrounds and includes us through sensory means; in a sense, the extent to which our external sensory inputs are occupied with a given task vs. any distractions.

Presence is the measure of the degree to which we feel we are in or part of an environment; in a sense, the extent to which our internal sensory and cognitive systems are occupied with a given task vs. other thoughts.

These two concepts are closely related, but subtly different. In one sense, you can think of the difference as cause and result. When we want to create a feeling of presence, we may employ immersion as a technique.

Low Immersion

But in other ways, it is not such a strict cause-effect relationship. We can be immersed in a good book, or a movie on an old 12″ TV and be completely distracted by a cell phone inside the best VR experience in the world. But in general, the feeling of presence is enhanced by providing contextual cues, peripheral vision, ambient sounds, lighting, and so on. The more of those we provide (in a reasonable fashion) to the human sensorium, the easier it is to create that feeling of presence.

High Immersion

And with that feeling of presence, a person is more cognitively engaged. And, as a result, he or she may more effectively receive, and perhaps communicate, information. Very subtle cues can convey a great deal of information, simply because our brains are more engaged in their detection and processing. Some cues, like smell, go straight to the oldest (from an evolutionary perspective) and most powerful (from a communications perspective) regions of our brains.

Low Presence

Comfort, satisfaction, performance, understanding, and even creativity may all be enhanced with increased presence. The task of understanding and enhancing this feeling is one of the principal challenges to using 3D graphics for communication.

High Presence

Interactivity, Reflectivity, and Resonance

Interactivity refers to the degree to which a person can make choices within an environment. These choices, when present, are based on the rules and behaviors of the environment and should ideally mesh with our expectations.

Reflectivity (also reciprocity) refers to the degree to which a person can alter an environment based on their expressed intentions, and the degree to which the environment reflects the changes a person intends.

Resonance refers to the degree to which a person and the environment mutually effect each other and build upon each other’s responses.

Without interactivity, games would seem like sporting events or movies. We could watch and cheer. But we can have no real impact on the events and on what is ultimately communicated back to us. But it’s a bit more complicated than that. A good movie or sportscast does actually engage us in some way and cause our thoughts (at least) to interact, one-way, with what we experience. For example, a murder mystersy TV show may cause us to suspect one person over another. The writers are counting on that to make the next reversal have a more dramatic effect. So we are interacting, even with a movie. But our interactions are all internal and have no effect on the movie itself.

Reflectivity, then, measures how much our interactions and expectations can change the virtual world that is communicated to us. If a movie reflected our intentions, we would have some control over how it unfolds. Games tend to be more reflective. But even today, the degree to which we can “go off the path” or modify the environment is limited in many games. We can’t always break a chair, even though we think we should. We can’t always ask a question of a non-player-character, though we might want to. So the more reflective something is, the more open-ended it is, and the more we feel we have control over how we (or our character) act and interact.

Resonance is the desired end result. The more interactivity and reflectivity we experience, the more likely we are to react and enhance the process. In the cycle of communication, resonance can actually feed back and cause more presence, more interactivity, more reflectivity.

For example, say we are in a 3D environment and we do try to break that chair. The chair shatters, sending splinters in the air. We reflexively duck to avoid the splinters. That’s a resonant effect, because we chose to break the chair, without necessarily thinking about the next step. And the result is we may feel more present in the environment. If one of those splinters hits us and we are aware of the sensation, that’s another resonant effect, which may make us feel even more connected. We’ve gone through two effective cycles of sensation, action and reaction from one enabling event.

The key is to take that process and apply it to visualizing and interacting with information.

3 thoughts on “Web 3D, Part 5: Glossary

  1. Pingback: The Other Here » Blog Archive » Avi’s 3D Web Interviews, Game Over, V for Verbeck, Throwing Spaghetti at the Metaverse, Stephen Eagle Man!!1, SLoogle Earth Slashdot

  2. Pingback: RealityPrime » Holographic Video for Your Home

  3. Pingback: RealityPrime » What is a Virtual World?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.