The Augmented World


Some people have asked for some more elaboration on where, in my view, the fields of 3D worlds, Virtual Globes, and Social Networking are heading. I’ll share a few off the cuff thoughts. It’s not like I’ve ever thought about this much.

What I think Google and not enough other companies understand is that the question is not, "what can we do to make a product and make money?" but more simply "what do people need?" Google talks about this endlessly, knowing (I presume) that if you can answer that question, the money may follow. I wouldn’t even ask what people want, because focus groups and the like can only answer multiple choice questions. The set of possible answers must already be known.

So what do people need? Let’s focus on information delivery and making use of the world’s collected knowledge (otherwise known as contextual decision making). And let’s not constrain ourselves to the current players at first. Let’s jump ahead to some yet-to-be-imagined company in Web 3.0, or "Web 3D" as it may be called.

Let’s call the company, "Hippo, Inc." and we’ll call their development offices the "Hippocampus." Their mascot is the horse, naturally, internet of the ancient world. Or so as not to confuse the kids, we can stick with the river horse, aka the friendly neighborhood hippopotamus (also, if global warming is really bad, we’ll need the swimmers). Either one will do.

What do they make in the Hippocampus? They make memories, of course, and a bit of spatial cognition too. Their device, the Hippodome is capable of recording your living stream of memories, to the extent you wish, extracting the relevant knowledge and raw information. It can return those memories to you, independent of your natural mental abilities and in new associative patterns. But the real power comes in when you factor in everyone else’s memories too. They’re all in the system, mixing together, like glue (sorry, horse). Suddenly, the world comes alive with information, stories, details about every place, every person, every thing we come into contact with.

You meet a person on the street. You know them instantly, as if you’ve always known them. You go to a new city and you already know your way. You want to make a political point about the war in Fakestan and its entire history is there for you, already in your mind. Congratulations, you’re an augmented human, independent in individual free will (to whatever extent it naturally exists), but connected in memory and awareness of the world.

Now, it’s not the Metaverse. It’s not virtual, though it is often spatial. It’s something real, something I’d rather call RealityPrime–the real world, plus-plus.

And it has the potential to change the real world far beyond what the Internet has done. In fact, I could speculate that if Fakestan had Hippodomes too, the ethnic tensions a few spice barons exploited to start the conflict might somehow melt away in a cloud of understanding. Of course, if Hippo, Inc. is a corrupt belligerent monopoly or is otherwise beholden to oppressive authoritarian forces, then all bets are off. Like any technology, it can be used for good, or evil. That’s why the choices Hippo makes are so important.

Well, either way, this vision is still a few years out. But it seems we have the building blocks available today.

Google’s mission is to assemble, index, and distribute the world’s information. Certainly that will someday include our memories. I mean, isn’t that already happening with what we publish: our blogs, newspapers, and historical documentary sites? (well, to be fair, some sites do more dissembling than assembling, but that’s not all that different from how our memories and reality-filters already work).

Google Earth and the like, especially on mobile devices, provides the on-the-spot situational awareness for the everyday jockey. The fact that it’s displayed in cool 3D graphics is a result of the best pathway we currently have to our brains. The visual system is capable of turning a series of pseudo-3D static images back into some form of tempo-spatial cognition. it’s a miracle it works (see the image of Annie above). But someday, we’ll have a rendering engine that can render those concepts directly.

Microsoft’s SenseCam, Accenture’s Personal Awareness Assistant, or a few others seek to let you record your life, though on whose terms isn’t clear. As bulk information, it’s nearly useless. But as semantic knowledge, it’s a treasure trove that can probably describe you better than you ever could.

And places like SecondLife? Well, creativity is best not restricted to the real world, in my mind, though the intersection can be fun. But to the extent that Linden Research and others become brokers of trust in distributed spatial transactions, they may also have a role to play.

And let’s not forget MySpace (how can we ever?), which serves the same function as our clothes — to dress us up, to hide the naughty bits, to speak to the world who we are, or at least who we want to be. And it shows us who is like us, and who might like us anyway.

To the extent that privacy is at all respected now or in the future, by law or by tradition, we have levels of access to our innermost selves. First you get the avatar, the home page, the superficial ego. Then you get associate, or if no monetary transaction is involved, the friend. Eventually you find the lover–and I’m not speaking of exchanges of bodily fluid, but of vulnerabilities and trust. And finally, ultimately, you find the more spiritual true self, the person you get to know over many years. Except in RealityPrime, the information is all there, subject to the access we grant.

At least, I’d hope it works that way. There is a possibility that in a truly open, augmented world, where every bit of information is available on the first order, we would quickly realize who was out to scam us or take advantage. But my sense is that the criminals are always the first to find a way around any protections, so I hope we start opening things more slowly than the idealist vision might prescribe, if for nothing else then to gradually test these ideas in the real world. It’s already begun.

  1. #1 by Ira on July 13, 2006 - 9:59 pm

    Avi, I *love* your vision of the future computer/internet-augmented world, which is (suprisingly) similar to mine. Nearly everyone voluntarily carrying a future cell phone/PDA/internet device that superimposes 3D graphics and text information on our field of view and audio information to our ears such that we have nearly instant access to whatever we need to know – the time, the weather, maps, directions to where we want to go, movie and train schedules, and so on! In particular, you say: "You meet a person on the street. You know them instantly, as if you’ve always known them." How do you expect that to work?

    [Long essay about the benefits of RFID for PositiveID edited for length by avi — please keep it short — it’s bad enough people have to hear my long-winded opinions. But you might want to consider linking to your own blog post for anything that long.]

  2. #2 by avi on July 14, 2006 - 7:29 am

    With RFID, the power goes to the collector. If I were designing such a system, I would give control to the individual over when and where their information is collected, how long it is stored, and who has access.

    What RFID (and this mythical Hippo) needs is a programamble communications and information "rights" policy per individual (opt-in) and a big ON/OFF button. That would go part-way to having some security and individual control. Right now, if I use an EasyPass in a car, it’s always on. If I want to pay cash, even if I stick the thing in the glove compartment, I run the risk of the EasyPass being detected accidentally. There is no ON/OFF switch.

    But right now, I can choose how much information I put on the web and to some extent, who I give access to it. That principle is critical for maintiaining individual autonomy, where RFID fails the most basic tests. It’s a technology designed for boxes and cattle, and that’s how it treats people.

    The thing that gets hard about Hippo is that information would be out there about you that is not authored or controlled by you, just as web pages might write about you today. That’s a fact of life of having any public exposure (that, plus libel laws). But collective ratings systems like Slashdot’s Karma and Wikipedia will go a long way towards weeding out the bad/erroneous information and bumping up the good.

  3. #3 by Ira on July 14, 2006 - 2:37 pm

    Avi, sorry for my long essay. However, in your reply, you neglected to tell us how you *would* achieve the part of your vision that said: “You meet a person on the street. You know them instantly, as if you’ve always known them.” How do you expect that to work?

    From your reply, it seems the meet and greet would only work if that “person” (presumably someone you have never met before) had decided to opt-in and provide full access to all his or her information by *anyone* (or at least by some class that happened to include you). I guess I just don’t understand what you meant and how “Hippo” would work. Do you think most people would opt-in at the maximum level? If not, paraphrasing your original statement: You meet a person on the street. They remain a total stranger, as if you’ve never met them.

  4. #4 by avi on July 14, 2006 - 4:05 pm

    Ira, it’s a hypothetical company. We don’t have to solve the engineering problems in blog comments.

    As for policy, I’d hope people would be able to trust in strangers enough to publish their personal info so that when the stranger meets them, they receive access to enough layers to be meaningful. But yes, if someone chose not to opt-in to the system, they’d be a stranger still. If you want trustworthiness, build a system that encourages trust. No RFID system can enforce that from above.

  5. #5 by Ted on July 15, 2006 - 2:31 pm

    It seems to me that the biggest building block that’s missing is the direct neural connection that your scenario relies on. It’d be relatively easy for a pair of goggles to display search results or a wikipedia entry on every person you meet, but having that information — let alone the memories that others have of that person — injected into your brain like your own memories is an altogether different matter.

  6. #6 by avi on July 15, 2006 - 2:36 pm

    Agreed, Ted. But that’s the point, looking to the future and then coming back to what’s practical. We can do most of the other stuff using visual media. The direct interface is ideal in many respects, but isn’t required. So why hasn’t anyone done this yet?

  7. #7 by Ted on July 15, 2006 - 7:30 pm

    You mean, why don’t we have goggles acting as heads-up displays? I’m guessing it’s a hardware issue. Right now any hardware that could act as a decent display is too bulky to wear in public. For years people were willing to carry about cellphones even if they were bulky, providing a demand that drove miniaturization. But until technology and fashion intersect on the matter of wearing oversize goggles, there isn’t a demand to drive further miniaturization.

    Plus, there’s the need for a critical mass of content. Unless enough people wear RFIDs, there’s insufficient incentive to wear the goggles. People might use goggles to view non-customized information (like ordinary web pages) during face-to-face interactions, but currently it’d probably be considered rude to do that.

  8. #8 by avi on July 16, 2006 - 8:53 am

    I think even if the device was hidden, like in our VR contact lenses, it would be rude to pause in the middle of a conversation to assimilate some information. Kind of like when people have bluetooth headsets on and they talk to you face-to-face and probably someone else on the phone.

  9. #9 by Ira on July 16, 2006 - 4:49 pm

    Ted’s comment “Unless enough people wear RFIDs, there’s insufficient incentive to wear the goggles” triggered a thought on how we could achieve part of Avi’s Hippo vision *without* RFID. This applies to the embarrasing situation of meeting one of the thousands of people you have met in the past but not being able to recall their name, occupation, etc.

    Imagine a future cellphone/camera/PDA light enough to carry in your shirt pocket. Whenever you meet someone, the device creates a voice to *text* record of their name and other information they voluntarily disclose to you. It also creates a voiceprint and a face geometry record. (It would *not* keep a recording of their voice or their photo – just the info text, voiceprint, and face geometry record.)

    If you meet one of these people in the future, your PDA would compare the new voiceprint and/or face geometry with all stored records and the system would display their name and other info in your VR goggles.

  10. #10 by Ted on July 16, 2006 - 7:24 pm

    Kind of like when people have bluetooth headsets on and they talk to you face-to-face and probably someone else on the phone.

    And yet people are doing that more and more. Cellphones have created behavior that once would have been considered unforgiveably rude, but is now increasingly tolerated/accepted. The adoption of wearable heads-up displays will rely on a similar change — or deterioration — in social standards.

  11. #11 by avi on July 16, 2006 - 11:31 pm

    It gets pretty messy. Aside from the social expectations, there’s a bunch of liability concerns. I give it nine months before the first Nintento Wii lawsuits about injuries from using the controller (bumping into furniture, etc..). It’s one of the things that has held HMDs back, and it’s a similar issue for portable TVs on eyeglasses.

    Right now, you can kind of tell when someone is talking on a cell phone because they get that distant stare in their eyes. But once they’re actually seeing the person they’re talking to, and everyone else in the room isn’t, how can we distinguish between that and schizophrenia? I mean, I remember walking in Seattle and some guy was having a lucid conversation on his cell phone, except the phone was a crushed plastic coke bottle. It only gets worse.

    I doubt I need to mention the incidence of strangulations of people using “Push to Talk” and speakerphone (well, none that I know of, but I’ve contemplated it myself). I expect more of a backlash coming.

  12. #12 by avi on July 17, 2006 - 2:27 pm

    By the way, Ira, on the idea of everyone wearing RFIDs to identify themselves, there’s a much better way. In the next few years, everyone’s cell phones will have GPS. All we need is for the GPS phones to transmit their location to a server (opt in) which can identify people based on their position alone, given an accuracy of 1-2 feet and speed and heading, at least outside.

(will not be published)