Category Archives: News

On Holographic Telepresence

by

2

Alex Kipman, the head of the Hololens project at Microsoft, gave a talk on the TED stage where he showed off some fantastic new Hololens demos and talked about the future of the technology.

Source: Check out Alex Kipman’s mesmerizing HoloLens TED talk   [I’ll replace the low-quality video with the official TED video when it’s out]

I am almost-beyond-words excited to finally see Microsoft showing off its holographic telepresence. Immersive games in your living room are amazing. Enterprise apps will be empowering. But Telepresence is the killer app for AR, hands down. It’ll still be some time before this replaces video chat and phone calls for everyone, but that time will pass quickly.

Holographic Telepresence is the main reason I pushed so hard for our small Analog Labs team to pursue head-worn displays (Screen Zero, as it was originally called). The original prototyping work behind the very first patent on this started a few months before I joined Alex’s team, with a simple demo of me as a video-quality “Star Wars” hologram, showing a better way to communicate.

Over in Craig Mundie’s org (SBG), we’d been trying to develop Avatar-based telepresence up to then, culminating in Avatar Kinect. I learned just how far we are from making avatars good enough for every day use. [Sorry Faceblock, it’s going to be a long while to wait. ]

Just as I joined the amazing team of PMs, designers and engineers in Analog Labs, we were fortunate to be asked by upper management to re-imagine the next Xbox console. We were still in the last nine months of shipping Kinect. Alex, Ryan, Mark, Johnny and others were heads down, working super hard on finishing Kinect.

For our top secret Screen Zero/Fortaleza, it was actually perfect timing, inside a company notorious for in-fighting and killing inspired ideas — unless of course they could show billion dollar values early on or directly support the mothership (Windows/Office). AR was not nearly at that point of confidence. Far from it, it was more likely to cost a billion than earn it anytime soon. (I’d estimate 1-2 billion spent thus far).

Needless to say, at the time, AR was a giant ball of risk. Few people understood the potential or the actual time to market. In fact, Alex was originally going to pursue autostereo 3D televisions as the “next big thing,” a kind of follow-up to Kinect. But after seeing six months of concepting, prototyping, and demos for Screen Zero, even Alex was convinced this stuff will be the future. It was just a matter of time and money, both of which Microsoft had.

I give Alex so much credit for getting the project to this point. In my opinion, he was the only person I’d met in the entire company with the political skills needed to keep HoloLens alive long enough to see daylight. It’s difficult to even describe the kinds of “House of Cards” maneuvers he had to do to remain in control, gather more resources (people & tech) and prevent disruption from executives and even former executives who either didn’t get it or had other ideas.

I may get around to telling the rest of the story some other time. But I just wanted to say how proud I am of the team and the vision to show the world a glimpse of our collective future.

Amazon Update

by

2

Back in 2014, I published a post on why I rejoined Amazon, after spending a year at a startup. I’ll say again that Amazon is the best run company I’ve ever worked for.

I say that after close to 25 years spent with many big and small companies. I could be getting a startup funded right now. I could probably make more money doing something else. I’m at Amazon because I want to be, because it’s an amazing place to work.

The thing I tell everyone who may be thinking of coming here, perhaps after reading slanted commentaries, is to read the leadership principles yourself. They really do represent the company and what it’s like to work here. And these LPs evolve with time and input as well.

The bigger news is that I helped spin up an exciting new project in my free time and now we’re growing. We need one great back-end developer and one great UX designer to join an amazing team. You can email me, or apply at the links below. I can’t say what it is, but it’s fun and it’s new and it’s like nothing you’ve ever seen before.

http://www.amazon.jobs/jobs/341935/senior-software-development-engineer

http://www.amazon.jobs/jobs/326804/interaction-designer

Nintendo Fires Employee For Speaking On Podcast

by

0

Last week, Nintendo localization editor Chris Pranger made an appearance on a small podcast called Part-Time Gamers. This week, Nintendo fired him.

Source: Nintendo Fires Employee For Speaking On Podcast

As companies acquire more and more (reportedly constitutional) rights to free speech, employees of companies ironically lose them.

When did this happen and who said it was OK?

Sure, “at-will” employment anticipates companies will fire and employees will quit for no apparent reason. On the other hand, one has to wonder what’s wrong with companies that treat their employees like worthless cogs vs. human beings whose passion and creativity are what make any (technically lifeless) entity seem alive in the first place.

You would think these companies would encourage that passion, but perhaps spend a few bucks up front to train everyone as to more media savvy techniques: when and what to say vs. nothing at all.

I have personal experience with this, having almost been fired by Microsoft (Balmer-era, to be fair) for blogging. Later on, I apparently had a promotion temporarily held hostage over it.

Granted, writing about the company you work for is never to be done without support or counsel. Writing about what they should do in the future is extra-ballsy, perhaps even nuts. In my case, I actually had prior permission from two more senior employees in my org (one being my boss, both of whom defended me) to blog about the topic. And yet that was almost not enough to stem the tide of an angry mob escalating the issue to the top.

“To the cannon!”

Some may say that what I said was going against company direction. Others know that I was being extremely diplomatic, even generous, in my public comments compared to reality.

The VP who most frothed at the mouth over my post is now out of his. His product is now radically changed for the better. And what I’d said was such basic common sense, like “level duh,” that it eventually took hold and became overall company policy, no thanks to me.

What I know for sure that I was ridiculously naive about the reaction of the press. Sites like “The Register” were so eager to bash Microsoft that they were willing to use me as the tool, instead of actually reporting. Because reporting would have required effort, at least a little, to figure out the really newsworthy story under the hood.

So the moral of this story is not that “one can’t talk” — we do in fact have free speech as long as we’re not afraid to use it — but that one can’t talk in such a way that it gets significant media attention without buy-in from suitably higher and higher levels of the company in advance.

Because the more attention you get, the more powerful people in a company will rise up to protect their own vested interests. And that, in the end, is why people get fired for talking.

Microsoft Hololens

by

0

I am proud and excited that Microsoft has finally announced a project I started working on in 2010, before anyone called it “Fortaleza” or “720” or “Hololens”. When I started architecting AR systems and designing the very first experiences around what we first called “Screen Zero”, I didn’t care about credit (lucky me, because I wouldn’t get any). I just wanted to help change the world.

And so it will…

One of the most fortuitous aspects of this is that I now get to work (at Amazon) with two of my original teammates from the early Screen Zero days: Rudy and Sheridan. Andy is still at Microsoft, maybe Katie too. But there are many more veterans working with us now. Amazing people make amazing things!

VR Hackathon 2014

by

0

I had a lot of fun helping to judge several dozen lively entries in the SF VR Hackathon this Sunday. I guess I’m at a point in my career where I’m qualified to judge but don’t have enough time to actually create fun projects (outside of work).

For “only” three days of work, there were some amazing entries. A number of them won prizes. I wish we’d had more categories and prizes to give out to some of the other notable efforts, like a mind-bending recursive/immersive zombie game from one of the sponsor teams and a really interesting virtual world built entirely in a pixel shader.

The grand prize went to a ghostbusters riff that did an amazing job solving (for narrow use cases) user input in VR, which I think is still one of the biggest unsolved challenges. Here’s some video of their experience. The controller had great haptic feedback, and the weak cardboard backbone connecting the two pieces actually added more value than it took away.

via Who You Gonna Call – VR Hackathon 2014 – YouTube.

The New Rift

by

2

This post by Peter Berkman gets closest to the meat of our collective concerns over the Oculus sale to Facebook. John Carmack even responded in the comments, while Palmer took to reddit. Raph Koster and Blair McIntyre do pretty good analyses in their own rights, putting the pieces together.

Disclaimer: I know many of the people mentioned here and consider them friends. I’m sticking to only what is publicly revealed information. And I am of course happy for these individuals who are getting to work on dream projects and making lots of money as a side benefit.

Here are some collected facts that are out in the open:

1. Industry Legends John Carmack and Michael Abrash have each talked publicly about wanting to build the Metaverse (of Snow Crash fame). I totally get what’s in this deal for them: build it with the right tech, the right people and at Facebook scale. Got to love the opportunity to realize that dream.

2. Cory Ondrejka (kick-ass VP of Mobile at Facebook) was formerly CTO of LindenLab/Second Life. He’s built a slightly less scalable version of the Metaverse already. Notice a “Mark, Chris and Cory” named in Palmer’s blog post. I have no doubt they’re completely sincere about wanting to see VR succeed in its own right.

3. Peter is also right in assuming that Oculus will add real gaze tracking. Palmer hints (skip to just before 19:00) about it here. Oculus is not alone, I expect. While there are many patents (of various quality) on this, there are still many viable ways to determine a user’s gaze, and many reasons to do so. Facebook has even more on its wishlist, I bet, beyond the basics of providing better visuals or more Natural UI.

4. Companies like Tobii make their living today by (effectively) enabling mind-reading via a user’s gaze. Your supermarket shelves were probably arranged with this kind of tech strapped to willing users. Your favorite website’s layout was probably tested using this tech in a lab. Just imagine the power of knowing what everyone is looking at and why, of being able to read subtle emotion in the face, including those micro-expressions that always reveal the truth.

5. Blair is also right that Zuckerberg’s AR long-term vision isn’t a typo. For all the arguments about VR vs. AR vs. camera-based vs. see-through being better for this or that, the bottom line is that VR sucks for mobile. That’s based purely on the old “wall meets face” principle, if not the geek factor of walking around talking to people with a brick over your eyes.

Facebook needs to win on mobile. Yes, it’ll take a few years just to get consumer-friendly seated VR right. But when Oculus eventually gets their glasses down to sunglass size and adds forward-facing cameras that mix real and virtual, then watch out[side]. Here’s the key part of Zuckerberg’s quote again:

But this is just the start. After games, we’re going to make Oculus a platform for many other experiences. Imagine enjoying a courtside seat at a game, studying in a classroom of students and teachers all over the world or consulting with a doctor face-to-face — just by putting on goggles in your home.

This is really a new communication platform. By feeling truly present, you can share unbounded spaces and experiences with the people in your life. Imagine sharing not just moments with your friends online, but entire experiences and adventures.These are just some of the potential uses. By working with developers and partners across the industry, together we can build many more.

One day, we believe this kind of immersive, augmented reality will become a part of daily life for billions of people. [emphasis mine]

6. I’ve heard Oculus folks also talk about telepresence as one of the killer apps for VR too, esp. when one can solve the “eye gaze” problem. You’ve probably experienced that in crappy modern video chat, which is generally how I’d imagine women feel when you stare at their chests. Interestingly, Cory’s old boss from Linden, Philip Rosedale, has announced he’s working on the eye gaze problem in his new startup too, and using some cool software called FaceShift to get a jump on the harder problems there.

[For full disclosure, I did some work on avatar-based telepresence as well, but I wasn’t entirely satisfied with the result. FWIW, I found a better approach, which works with no hardware sitting on your face.]

7. Finally, Facebook makes its money by selling its users. Let’s not be coy. It’s a lot of money and a lot of users. To be fair, is it really any worse than how NBC offers up its users to advertisers (since before I was born)? Well, no and yes. It’s the same “ad, ad” world as before. We learn by age four to distrust the loud man on the TV. But the key difference is that NBC can’t see you, can’t really know your thoughts, except via gross statistics (think Neilsen ratings). Facebook really wants to know you, individually, and forever.

Now, companies like Facebook and Google provide immensely desirable services for free too, more so than NBC IMO. The problem lies in the concept of “informed consent.” Both tech companies still suck at the human stuff IMO, esp. in terms of giving users clear information up front to make that decision wisely, and then giving full control of their data after. It’s still mostly one-sided today, and it has to change.

Ideally, they’d start to emulate companies who’ve adopted an obsessive customer focus. It’s not about what people will tolerate, but what they really need. Earning our trust can’t be an afterthought or a win by attrition.

[for a positive comparison, TV’s “Neilsen families” give truly informed consent and the company promises not to market to them as well.]

So we come back to Peter’s insightful post. The problem with the Oculus sale is not that we all supported Palmer’s fortuitous kickstarter and got no share of the spoils. I got the product I ordered, still sitting in its box, alas. It’s not that he and Brendan Iribe “sold out” the indie gaming dream for big bucks. There are plenty of reasons to seek this level of protection and financial support in the light of Sony (and others) gearing up for a big fight. Gamers should be happy to have more viable options to choose from at the lowest prices.

Hell, I joined the so-called “evil empire” in 2008 to try to use their massive scale to do some good in the world, so who am I to judge? (jury is still out on my contributions, fwiw)

No. The heart of the problem is that VR is the most powerful means ever invented to pipe external ideas into your internal world. It’s pushing remote-controlled information almost directly into your brain via every sense possible. Consider that people believe a lot of crap they can’t even see. You’ll believe this crap on so many levels. That’s why it’s called virtual reality, ok? It’s as close to real as it gets, while still being entirely artificial.

The Facebook purchase highlights the reverse of that flow as well, reminding us that VR may become one of the best ways to pipe your internal world out — via data mining, classification and onto untold dissemination. That’s actually one of the reasons I got into the field, 20 years ago: to more easily tell stories that were rattling around my brain that would have taken millions of dollars and hundreds of people to produce as movies or games. But it has a dark side too, like when the information is sucked out of us without our informed consent or control.

Just think what the headlines would have said last week if Facebook had instead bought a brain-sensing startup like Emotiv, or invested in fMRI brain scanning tech to extract your thoughts. See what I’m talking about? VR doesn’t work quite the same way as those, but it gets to the same place in the end. The protective walls between you and the world come down in favor of more bandwidth in and out.

So closing that loop, even crafting individually designated virtual worlds using all of your private information, Facebook will own the most potent means available, short of mind-control drugs, to read and write to your private inner world, your thoughts, your actions, your dreams. It can free you, or it can enslave you.

What they do with it is entirely up to them (plus certain market and “other” forces). And if that doesn’t scare you, at least a little, then you may already be sold.

 

Bing Maps Preview Arrives

by

0

Big Congrats to the teams in Bellevue, Sunnyvale, Boulder, Graz and more.

What you’re seeing here is the product of 10 years of effort, in many ways, and represents a turnaround and partial recovery from one of Ballmer’s biggest mistakes. He famously handed most of the company’s mapping business to Nokia, losing many smart minds and multiple years of effort in the process.

But a core group remained, and with the company’s renewed support were able to build this as a first step to catching up. And it’s a very admirable step. Nice work!

From Bing Maps Preview Arrives on Windows 8.1 – Search Blog.

Fukushima

by

0

This is the thing about nuclear power. It doesn’t just go away when we’re not looking.

http://www.huffingtonpost.com/charles-perrow/fukushima-forever_b_3941589.html?utm_hp_ref=science&ir=Science

Long story short, fuel rods are waiting to be removed to a slightly safer location. Company doing the removal is probably going bankrupt and the government is probably incompetent. If the rods touch, here comes Zuul. We might not have a Tokyo anymore. And in the meantime, cesium detectors might be a useful addition to sushi bars, especially the good ones that fly in their fish…

If you’re thinking, “hey, what if there’s an earthquake somewhere near where I live and the same thing happens again?”

Good thinking! Here’s a handy map of nuclear plants and earthquake zones:

http://maptd.com/map/earthquake_activity_vs_nuclear_power_plants/

Which makes me think that all those people prepping their little post apocalyptic shelters against evil bandits (after the downfall of civilization, or Obama’s third term) might want to rethink their security plans.

If civilization goes down, so do the people maintaining these nuclear plants. So do the deliveries of fuel to keep the water pumps running. We’d be relying on plants whose fundamental safety plans may be based on the assumption of everything else working just fine.

Personally, I’d be more comfortable with nuclear power if the plants could demonstrate that they’d be the last ones standing (and still generating power) after any number of catastrophes outside.

Otherwise, we might as well draw a map of every nuclear plant in the world being a massive emitter of radiation at once and note the Extinction Level Event we built to power your son’s night light.

Or, you know, we could fix it. We have plenty of time and ample cause for action.