I got my professional start working in Virtual Reality in 1992 for a small start-up in Seattle, where we built big immersive displays from wood and plastic. Later, at Disney, I got to work with both custom head-mounted-displays and wall-sized projection displays (we built the world’s first hexagonal CAVE, a good 5000 pixels across). I’ve researched and endlessly pondered the issue of where display technology is heading. Let’s see if my answers are in line with yours.
The State of View
Right now, there’s a bit of disconnect between what’s marketed to consumers and what’s best. Many of you have already gone out and bought your swanky 42 inch or better plasma/LCD/DLP TV sets. Better make it last 4-5 years. From 5 years out, the "must have" will be wall-sized displays and you’ll probably want to shell out once again. Fortunately, you can’t go much bigger than a wall without rebuilding your house.
For those willing to do a little home engineering today, ceiling-mounted projectors are bright enough to easily scale to 10 or 12 foot (180 inch diagonal) displays without dimming the lights. At reasonable price-points (say, the price your LCD TV) resolution is limited to 1024×768 (e.g., 720p), but full HDTV resolution and higher starts at around $8000 today and will be down to the low thousands within 3-5 years.
With HDTV still slowing coming to life, the next trend will be immersion, not more pixels (the last thing the industry needs now is another standards fight). Larger screens (10 feet and up) are a decent substitute for the movie theater experience. But for immersion, you’ll want some peripheral vision, aka wrap-around with at least 1/2 the resolution of the center screen. And you’ll have two main options. Projection can always fill in in a pinch, using a curved screen with software or corrective optics or using 3 flat/angled screens. Three projectors would be fitted into a single "emitter" device and the screens may even fold to save space (but increasing cost). But get ready to reconfigure your living room in any event. The simplest possible version of this sees the "ambient lighting" colored-LED gimmick taken to its logical conclusion — two small projectors at the edges of your TV project additional peripheral color information onto your wall (or two slide-out screens) at some resolution, extending your TV significantly.
We already see the start of this with multiple-monitor setups for your PC games. It’s still niche, but it’ll go mainstream once the networks (especially cable) realize they can use their spare bandwidth to offer extended panoramic views as a premium option. In other words, they’ll program for 3 screens simultaneously, the main center channel and two optional sides, all linked to form a seamless panorama.
Watch for this in sports first. In fact, this trick could be done today, perhaps in a sports bar with three big TVs or a theater with a single very wide screen. The main technical obstacle for the networks is in bundling 3 cameras on one pivot (not hard) and dedicating the bandwidth. There are some trickier issues, like how to handle "zoom," but those are still solvable with software or are at least avoidable.
The more exciting displays will come with any of a dozen variations on the flexible display surface, using electronic ink or any suitable medium such as flexible OLED. On the 5-15 year horizon (depending on many factors), you can expect to buy a "roll" of active display material, say 7 feet high and 20 feet long, which you can arrange anywhere from flat, to a semicircle around your couch, to a full 360 circle of video around you, standing in your immersive environment.
The current best choice for the 3D effect is the parallax barrier, essentially just another opaque display in front of your color screen that makes it so each eye gets a slightly different picture. That should scale to flexible technology as well, so [thicker] 3D wallpaper isn’t too far-fetched. And there’s always the good old lenticular lens (bending the light rays so each eye sees every other pixel). These are still expensive to produce, but with holographic lenses (photon-scale prints which mimic the properties of real lenses), the price and thickness may come down to nothing.
Head-mounted displays, even when shrunk to the size of sunglasses, are suboptimal for entertainment, mainly due to the inevitable latency when moving or turning your head. It feels a lot like being drunk, even when it’s only 1/30th of a second. Fixed (projection or any equivalent wall-sized) displays only have head-tracking latency issues when significantly moving your head, not turning, and you can get away without any head-tracking at all as long as you don’t need to interact with virtual objects closer than about twice your screen size. Adding parallax barriers to fixed-mounted displays, the latency can be dropped to zero, as long as enough channels (or slices, ready for each eye to see) are predicted and rendered in time to see.
But HMDs have one big advantage in portability. Augmented reality is the overlaying of virtual information on the real world. And it’s already used, for example, in aircraft maintenance. For entertainment, imagine a melding of the real environment with the virtual in alternate reality games, dragons in your back yard. This area is having some trouble commercializing right now, but the goal is to have a heads-up display for anything you might find today on your cell phone or PDA, including automatic recognition of objects, people, and places. At some point, your phone call will include a life-sized image (a "ghost") of the person on the other end of the line, walking beside you down the street (I’m ignoring how to get that image, just how to display it).
But that’s still a ways away. I imagine HMDs will become more common in 5-10 years with further miniaturization. Some companies are even using lasers to beam the image into your eyes without a screen at all (I tried it 10 years ago in the lab and I hope it’s improved). But the big sea change will happen when miniaturization takes HMDs down to the size of contact lenses in 10-20 years (certainly needing on-chip lasers or holographics to make it work) . There, you’ll see a dual interface. Open your eyes and you get an overlay of virtual on real objects, text, highlighting, even magnification. Close your eyes and you’re in a complete virtual world.
The more mundane vision is of the virtual office, which people are pushing for today in the same way they pushed for the paperless office in the 90s. Sure, I’d love to have a real holographic desk surface full of virtual 3D widgets I can slide around, virtual paper instead of monitor, Google Earth floating on my desk — that sort of thing. But will the benefits be worth the expense? Not for some time to come. Studies have shown that more screen area can equate to higher productivity (up to 30% iirc), but I haven’t seen any study that shows any benefit for a virtual floating piece of paper vs. a 2D window on monitor. I’m a big fan of 3D desktops, but mainly for aesthetic reasons. It’s going to take some enterprising software development to find a real use for the 3D desktop.
So for the easiest path through all this chaos, I’m imagining the next push will be for multiple or simply bigger screens, possibly multiple channels of television side-by-side followed by wrap-around displays for the home. In 5-10 years, we’ll see augmented reality take off for the office and the street. And the virtual contact lens brings it all together in a nice little package by the time I’m ready to retire. Which is a good thing too, since I may really need glasses by then.
See page 2 for further thoughts. Please feel free to comment, share your thoughts, and I’d be happy to go into more depth on any given area in future posts. This stuff used to be my bread and butter as a consultant, but I’ve moved on to other areas of R&D these days.