Rendering Synthetic Objects into Legacy Photographs from Kevin Karsch on Vimeo.

Congrats to the entire Google Geo team. Well done.

I came across this incredibly well written article at New Scientist, which touches on some of the possibilities. I won’t even excerpt it. Just read it.

This in a sense gives people yet another super power. The ability to read minds, or at least emotions. People with extremely high emotional IQ can already do this, but it eludes many others. Some people, like me, have high perception and sensitivity of other people’s states, but poor recognition of how we ourselves come across.

Technology can help this. And coupled with something like AR contact lenses, this can be a game changer for interpersonal interactions.

Forget the cliche of floating someone’s name or birthday over their head so you don’t need to remember it. What happens when computers guide our conversations to be more effective (appropriately affective)? Think business negotiations, social situations like dating, and basic training wheels for emotional interaction from childhood to adult.

The downside is a normalization of unique differences between people. But perhaps if we’re smart, we can retain the best of social interactions and improve the pitfalls without homogenizing everything to a too-simplistic algorithm.

Very promising indeed.

Someone at work forwarded me this comic, nicely designed to get you thinking about the following point: At what point does a game system evolve into an organized religion?

[And by religion, I don't mean it in the grand tradition of personal and spiritual belief systems, but rather as "institution designed to defend itself against other religions and the lack-thereof, and thus self-propagate."]

Think about it. Organized religion uses reward systems daily and in the afterlife too. Such an such sin gets so many hail Mary’s. Blowing up civilians gets so many virgins. it’s all about points and expectations, even if the promises are much less immediately gratifying than they might be in games.

In games, we can give you badges for achievements. We can unlock content you would not otherwise get to. We can promote you on social ladders and give you the attention and recognition you think you deserve.

What I find most interesting about this is that in thinking for a long time about real world game systems that the comic is skewering, I came to the conclusion years ago that there must be more than one such system. People can then subscribe to the system of their choice. That would lead to politically-driven point systems, religiously-driven point systems, and so on. And it’s not inconceivable that someone would try to make such a sociopathic point system eventually, intolerable as it would be. [this is where the rest of us have to use the criminal point system to stop it]

The main twist I wanted to offer on this meme is not just that we could see multiple systems in play, but that the ratification of the systems — maybe even the validation of the point awards themselves to some extent — must come from the participants, not some holy temple of rightness.

That doesn’t prevent mob rule, as is true of Democracy, but it at least guarantees that people can self govern, and if not, they get more or less what history has proven happens when people give up those rights or have them stripped without a fight.

Very interesting stuff. I wonder what my old Disney colleague Jesse Schell thinks of all this…

It was mostly enjoyable, except for some rampant anti-semitism in my house (e.g., people throwing pennies at us; two and one-half wandering Jews) mostly as a way for the upperclassmen to pick on the lower. It could have been anything, I guess. I didn’t let it affect me.

However, that mild abuse was a mask for something deeper going on. A few months after I joined, my formal initiation consisted of about a week of sleep deprivation, truly disgusting and unhealthy living conditions, coupled with some physical activities that literally left scars on me for a decade.

I almost bolted towards the end of it. But the upperclassmen threatened my fellow initiates with even more pain if I did, and so I stayed and made it through.

Then it was all beer and fun for a while. We forgot the pain and remembered that we’d made it through together, kind of like war vets, but with much less danger or purpose.

Fast forward to the next fall. I was a full brother and in charge of recruitment (“rush”) for a house that had declining membership and income. I had the bright idea that if every brother simply picked two freshmen they liked, spent a lot of quality time with them, and at least one of those two joined the house (on average), we’d do pretty well.

My plan worked a bit too well. We practically doubled the size of the house that year. And yes, we were one of the few houses to have a truly dry rush and still do well.

But I hadn’t ever thought about what it would be like to be on the other side of the hazing pattern, doing the deed to the new friends I’d indirectly brought in.

I left half-way through the initiation that year, distraught and unable to proceed. It was far worse to give than to receive.

I decided to end hazing in my fraternity for good. And fortunately for me, a few of my close friends (from my class and the one I’d help recruit) agreed to help.

We spent the next year on this task, sometimes to the detriment of our classes. We tried to get ourselves elected to lead the house. We lobbied to change the rules. We even drove cross-country to the national headquarters to research other local chapters that might have solved this already.

The pro-hazing forces were more or less on to us (the failed elections were kind of a tip-off) and came along just to watch us, thinking we were going to rat them out to the national leadership. It was quite a caravan.

We didn’t rat them out. Had we wanted to blow up the chapter, we could have simply gone to the dean, the media, the cops. Anyone in a position of responsibility would have to take legal action for liability reasons and shut it all down.

Instead, we followed a precedent for a house that had ended rampant drug abuse. We got our own alumni association (who was mostly pro-hazing for nostalga reasons, but also pro-not-getting sued for responsible-adult reasons) to take over the house and fix the problems quietly.

It worked. We even had the brothers believing it was the alumni association’s idea in the first place, just to make it easier to swallow.

However, with all of my machinations and arm-twisting to end hazing, I’d certainly made a few enemies along the way. And in the end, partly because of the neglect of classwork and partly to protect my co-conspirators, I took the blame upon myself and soon transferred schools to Ohio State, where I got my actual degree, safe from the death threats and angry stares. [I'm not sure they would have done more than basic violence, but a drunken group rage was not unlikely either].

In the end, the house did pretty well from what I hear. No more hazing to this day. There was none of the doom and gloom, social and moral collapse, as promised by the pro-hazing forces either. Life goes on, just a little less harshly.

Hazing is such an ugly and unnecessary thing. Brotherhood is built from common experience, trust and respect. It’s not built from one class persecuting the next, or from those in power abusing that power to get their way.

But the key lesson was: don’t rat out the rats if you can avoid it, but fix the problem just the same — and don’t give up until you do. In the worst case, you may just fix it but make it hard for you to stay. And that’s okay. Life is full of twists and turns along the way.

I hope that’s helpful to someone else out there. It was a lesson hard won.

Funny as hell…. Hang on Woody!

http://www.zdnet.com/blog/microsoft/microsoft-to-work-with-joyent-to-port-nodejs-to-windows-azure/9802

Node.js + Azure is a powerful combination, letting us run the same JS/JSON in the browser, server and database with an event model that makes scalability a breeze. Good stuff!

I am incredibly excited that this is being officially supported. Go team!

Is WebGL actually harming your computer in any way? I doubt that’s a serious or credible claim. And, frankly, if Microsoft has taken a formal position against WebGL, no one I know got the memo.

It would be an unfortunate position for Microsoft to take, IMO, because it gives the impression that Microsoft runs away from security issues that require some modest technical mitigation.

After all, what is an operating system but a series of security apparatuses coupled with Hardware Access Layers and useful software development APIs on top?

WebGL has the latter two in spades and most of the former, but clearly needs a bit more assistance on the security angle before everyone is “all warm and fuzzy.”

Operating systems and security mitigation are what Microsoft is known for. It’s our bread and butter. Why would we run away from that challenge with such an alarmist attitude of “shut it off, shut it off, it might hurt me!”

I think we would face these potential threats head on, as we’ve always done.

I mean, the exact same graphics hardware that WebGL uses is available to native applications running DirectX or OpenGL on your PC and/or phone. Are we going to ban downloaded games because they might, in some universe of possibilities, harm our computer or cause us to, God forbid, reboot? No. At most we give a security warning that this .exe might be harmful and let you make the choice.

Are we going to ban Chrome and Firefox as “unsafe” if they continue to unabashedly support WebGL? Are we claiming those guys don’t care about security and aren’t working hard to mitigate any remaining issues as soon as possible?

No. That’s not a tenable position, IMO. WebGL will be running on my PC and yours, one way or another. Microsoft will need to deal with it. And more to the point, we can actually help make it much more robust if we engage instead of apparently running away.

Remember Plugins?

I was mainly disappointed in those posts because I recall vividly that it was Internet Explorer’s pioneering work with plugins (specifically ActiveX controls) that help build the rich interactive web as it exists today. Plugins created capabilities not found in browsers, even to this day. Flash is a native plugin. Silverlight is a native plugin. Google Earth, running in your browser, is a native code plugin. RealVideo, YouTube, and FarmVille would arguably not even exist without plugins (okay, that last one might have been a blessing).

However, ActiveX controls were, at one point, the primary vulnerability for browser-borne attacks on your PC. They are, after all, native code with hardware access that could run malicious operations, perform disk writes, read your personal data and plant viruses. Indeed the MSDN site on ActiveX controls begins with “An ActiveX control can be an extremely insecure way to provide a feature.”

Yes, indeed.

Somehow we survived the existential threat of native code plugins taking over our PCs, or at least we made it through alive. The web prospered in rich user experiences primarily on IE, while the main residual downside of plugins, even today, is that they require user confirmation, code signing, and in some cases circulation of known or suspected threat information among browsers to help block attacks. That’s not ideal, but yet we survived and received the benefits of plugins on the whole.

Well, that’s not to say plugins are all safe. How often does your Flash plugin need to be updated (weekly?) to address vulnerabilities to keep it safe from attacks? If WebGL can help obsolete those security holes, it could actually be in some ways safer than what exists today.

WebGL is not a plugin but rather a “built-in” and it doesn’t ever allow the extreme native access of ActiveX — no disk writes, no main memory access, no CPU code apart from officially signed graphics drivers. A shader can really only affect your graphics hardware and screen output. The most severe vulnerability we know of today is that it might hang your machine. Worst-case solution: reboot.

We can do better. We can require WebGL shaders to be proactively trusted in the same way plugins are trusted and largely avoid the worst threats. We can do even better with code analysis, collaborative filtering, and hardware or OS watchdog  timers (e.g., any shader taking more than a fraction of a second can be reset without anyone complaining). Yes, we can. But if the choice comes down to running WebGL or not, I’d live with a popup asking permission to access my graphics hardware, as we do for GPS, camera, etc..

What’s going on?

From the one discussion I’ve had with leaders from IE, I can reassure folks outside Microsoft that this issue is actually about security and doing the right thing for users. It’s not about “GL” vs. “DX” in the name, as some suggest. It’s not about wanting to disrupt any other browsers, as Microsoft has often been accused. These leaders are genuinely concerned about the possibility that someone on a malicious website could use WebGL to disrupt your experience in a serious way, and incidentally that it would appear to be Microsoft’s fault…

Users are not very discriminating in their blame, after all.

Those leaders may not be fully aware of how big a movement WebGL really is and how it is going to transform the web yet again. But the reality is, if Internet Explorer does not support WebGL and WebGL nevertheless becomes the de facto standard for 3D on the web (which it will, IMO), then IE will be in an uncompetitive position to either help fix any problems and moreover retain or grow market share relative to other browsers. That would be sad, esp. given how long the product cycles are and how long it would take to course-correct. We could miss the boat entirely.

Now, I own Microsoft stock. I want Microsoft to succeed, and that includes IE. If Chrome, Firefox, and Safari support WebGL on Windows and there are new PC-only vulnerabilities found, do you really think people will blame Google, Mozilla, and Apple and praise IE?

Not a chance. They’ll blame Microsoft. They’ll blame the OS. They’ll blame the company. They’ll blame the logo sitting in the corner of the screen that just went blue or blank and say how “this never would happen on Chrome or OS X.” (ignoring market share)

All Microsoft would likely achieve by not supporting and improving WebGL securely is that the people who could really fix the few remaining issues (driver writers, hardware manufacturers, OS makers) will try less hard and take that much longer than they would with IE and DirectX demanding results.

Meanwhile, IE would potentially lose market share due to popular interactive experiences that are not achievable there. And any sort of weaker “safe” shader-less  alternative that IE might conceivably propose in a too-little-too-late DOA standard will make it appear as if IE is trying to disrupt the market, which I don’t believe is their goal. They really want to do the right thing, but it  may not be very clear what that is until the Web clearly and audibly demands it.

There is only one way through this maze. The way forward is to address the security issues head on, get IE the most robust implementation of WebGL on the market, and lead the industry to a new level of user experience, including NUI and rich 3D graphics, hand in hand.

Speaking only for myself, as always, I fully intend to use WebGL as one important tool for applications and platforms I develop for Microsoft. That means “wherever it’s supported.” For other cases, we’ll have to use creative fallbacks, lesser functionality, and/or resort to plugins or augmented browsers for cross-browser capabilities once again. Our charter at Bing requires working cross-platform to reach the greatest number of people possible and I don’t see that changing anytime soon.

The kind of experiences we want to deploy are nothing short of revolutionary – 3D for the masses, tying the real world  to the information space that surrounds us in our everyday lives. This means phones, PCs, and the like will require the kinds of rich, real-time interactive 3D interfaces that right now only WebGL can offer in a cross-platform, stable, browser-based way.

There is clearly only one direction forward for Microsoft and 3D on the web.

WebGL is the way.

Facial Expressions captured (3-5 years from 2008 = 2011-2013): True. Look at Avatar Kinect, coming soon. While limiting the rendering of said facial expressions to Rankin-Bass style XBox avatars was not my personal preference, the result still works well. The team has done an impressive job. However, it’s not as portable as I was hoping for in my original article in terms of the mobile AR solution, so a few points off for me.

Laser Retinal Scanners (2-3 years from 2008 = 2010-2011). False. Well, the tech apparently does exist, but it clearly hasn’t been commercialized to any degree, except in the form of pico projectors. Dim. Also, I’m now not at all sure this tech going to win in the end. See below.

3D Rendering is Photo-Realistic. (2011). True. High-end video game makers have been focusing on things other than realism in recent years. The result of their 3D engine work is certainly not indistinguishable from reality, but it works well enough not to notice. The physics of light and materials works really well. It tends to be things like cracks and dust and dirt that graphics programmers and designers overlook. The worlds tend to be too perfect, hyperrealistic rather than photorealistic.

Virtual Humans pass the Uncanny Valley (2013). Still possible, but doubtful. I’ve seen some examples of CGI still shots that could not only pass the uncanny valley, but easily fool a human into thinking it was real. I have not seen any 3D animation pass the uncanny valley though, except where it uses motion capture of a real human. And even then, it’s close, but typically doesn’t capture enough information to pass — the subtlety, deformations of fatty layers of skin and fluid dynamics. Even things like “eye gaze” is still a problem in 2011. For example, there’s good evidence we can accurately tell where someone is looking by the glint of their eyes, which changes based on the subtle deformation of their eyeballs as they go from “near” to “far”. Virtual actor “gaze” looked far more believable in Monsters, Inc. for example, than Beowolf. These are all solvable problems, but no one has put it all together yet IMO. Avatar mostly worked on this front, but it intentionally wasn’t using humans. We get two years until I need to admit the delay…

AR Contact Lenses (5-7 years from 2008 = 2013-2015 for sure; 2011-2013 is possible at great expense). It’s still possible! Well, forget the 2011 option, but this is looking better all the time. The University of Washington contact lens work is still 5-10 years out, alas. But the combination of a cheap mostly inert contact lens plus simple AR glasses is a potential game changer (assuming people who don’t need them will choose to wear the lenses). It’s certainly still possible for 2013, given Innovega’s recent public announcements. And Vuzix made some impressive announcements of their own from what I hear.

Google Earth for the Human Body (no date) — true! – with similar benefits and flaws we discussed. I don’t see the “mapping problem” solved yet, but it’s really hard. Give it time. As a teaching tool, it’s a great start. And it’s all done in WebGL as a bonus. I only complained about the sad state of Web3D standards, but that too may be about to change.

So as of today, I’d give myself about a 60% rating, with some items still TBD. Not a bad batting average for such aggressive estimates, but hopefully I’ll do much better next time.