Why Microsoft and Internet Explorer need WebGL (and vice-versa)


I was disappointed today to read the headline “Microsoft refuses to endorse WebGL, labels it ‘harmful’,” which itself is derived from a Microsoft security blog post titled “WebGL Considered Harmful,” which itself parrots a security scare report from a few weeks back.

Is WebGL actually harming your computer in any way? I doubt that’s a serious or credible claim. And, frankly, if Microsoft has taken a formal position against WebGL, no one I know got the memo.

It would be an unfortunate position for Microsoft to take, IMO, because it gives the impression that Microsoft runs away from security issues that require some modest technical mitigation.

After all, what is an operating system but a series of security apparatuses coupled with Hardware Access Layers and useful software development APIs on top?

WebGL has the latter two in spades and most of the former, but clearly needs a bit more assistance on the security angle before everyone is “all warm and fuzzy.”

Operating systems and security mitigation are what Microsoft is known for. It’s our bread and butter. Why would we run away from that challenge with such an alarmist attitude of “shut it off, shut it off, it might hurt me!”

I think we would face these potential threats head on, as we’ve always done.

I mean, the exact same graphics hardware that WebGL uses is available to native applications running DirectX or OpenGL on your PC and/or phone. Are we going to ban downloaded games because they might, in some universe of possibilities, harm our computer or cause us to, God forbid, reboot? No. At most we give a security warning that this .exe might be harmful and let you make the choice.

Are we going to ban Chrome and Firefox as “unsafe” if they continue to unabashedly support WebGL? Are we claiming those guys don’t care about security and aren’t working hard to mitigate any remaining issues as soon as possible?

No. That’s not a tenable position, IMO. WebGL will be running on my PC and yours, one way or another. Microsoft will need to deal with it. And more to the point, we can actually help make it much more robust if we engage instead of apparently running away.

Remember Plugins?

I was mainly disappointed in those posts because I recall vividly that it was Internet Explorer’s pioneering work with plugins (specifically ActiveX controls) that help build the rich interactive web as it exists today. Plugins created capabilities not found in browsers, even to this day. Flash is a native plugin. Silverlight is a native plugin. Google Earth, running in your browser, is a native code plugin. RealVideo, YouTube, and FarmVille would arguably not even exist without plugins (okay, that last one might have been a blessing).

However, ActiveX controls were, at one point, the primary vulnerability for browser-borne attacks on your PC. They are, after all, native code with hardware access that could run malicious operations, perform disk writes, read your personal data and plant viruses. Indeed the MSDN site on ActiveX controls begins with “An ActiveX control can be an extremely insecure way to provide a feature.”

Yes, indeed.

Somehow we survived the existential threat of native code plugins taking over our PCs, or at least we made it through alive. The web prospered in rich user experiences primarily on IE, while the main residual downside of plugins, even today, is that they require user confirmation, code signing, and in some cases circulation of known or suspected threat information among browsers to help block attacks. That’s not ideal, but yet we survived and received the benefits of plugins on the whole.

Well, that’s not to say plugins are all safe. How often does your Flash plugin need to be updated (weekly?) to address vulnerabilities to keep it safe from attacks? If WebGL can help obsolete those security holes, it could actually be in some ways safer than what exists today.

WebGL is not a plugin but rather a “built-in” and it doesn’t ever allow the extreme native access of ActiveX — no disk writes, no main memory access, no CPU code apart from officially signed graphics drivers. A shader can really only affect your graphics hardware and screen output. The most severe vulnerability we know of today is that it might hang your machine. Worst-case solution: reboot.

We can do better. We can require WebGL shaders to be proactively trusted in the same way plugins are trusted and largely avoid the worst threats. We can do even better with code analysis, collaborative filtering, and hardware or OS watchdog  timers (e.g., any shader taking more than a fraction of a second can be reset without anyone complaining). Yes, we can. But if the choice comes down to running WebGL or not, I’d live with a popup asking permission to access my graphics hardware, as we do for GPS, camera, etc..

What’s going on?

From the one discussion I’ve had with leaders from IE, I can reassure folks outside Microsoft that this issue is actually about security and doing the right thing for users. It’s not about “GL” vs. “DX” in the name, as some suggest. It’s not about wanting to disrupt any other browsers, as Microsoft has often been accused. These leaders are genuinely concerned about the possibility that someone on a malicious website could use WebGL to disrupt your experience in a serious way, and incidentally that it would appear to be Microsoft’s fault…

Users are not very discriminating in their blame, after all.

Those leaders may not be fully aware of how big a movement WebGL really is and how it is going to transform the web yet again. But the reality is, if Internet Explorer does not support WebGL and WebGL nevertheless becomes the de facto standard for 3D on the web (which it will, IMO), then IE will be in an uncompetitive position to either help fix any problems and moreover retain or grow market share relative to other browsers. That would be sad, esp. given how long the product cycles are and how long it would take to course-correct. We could miss the boat entirely.

Now, I own Microsoft stock. I want Microsoft to succeed, and that includes IE. If Chrome, Firefox, and Safari support WebGL on Windows and there are new PC-only vulnerabilities found, do you really think people will blame Google, Mozilla, and Apple and praise IE?

Not a chance. They’ll blame Microsoft. They’ll blame the OS. They’ll blame the company. They’ll blame the logo sitting in the corner of the screen that just went blue or blank and say how “this never would happen on Chrome or OS X.” (ignoring market share)

All Microsoft would likely achieve by not supporting and improving WebGL securely is that the people who could really fix the few remaining issues (driver writers, hardware manufacturers, OS makers) will try less hard and take that much longer than they would with IE and DirectX demanding results.

Meanwhile, IE would potentially lose market share due to popular interactive experiences that are not achievable there. And any sort of weaker “safe” shader-less  alternative that IE might conceivably propose in a too-little-too-late DOA standard will make it appear as if IE is trying to disrupt the market, which I don’t believe is their goal. They really want to do the right thing, but it  may not be very clear what that is until the Web clearly and audibly demands it.

There is only one way through this maze. The way forward is to address the security issues head on, get IE the most robust implementation of WebGL on the market, and lead the industry to a new level of user experience, including NUI and rich 3D graphics, hand in hand.

Speaking only for myself, as always, I fully intend to use WebGL as one important tool for applications and platforms I develop for Microsoft. That means “wherever it’s supported.” For other cases, we’ll have to use creative fallbacks, lesser functionality, and/or resort to plugins or augmented browsers for cross-browser capabilities once again. Our charter at Bing requires working cross-platform to reach the greatest number of people possible and I don’t see that changing anytime soon.

The kind of experiences we want to deploy are nothing short of revolutionary – 3D for the masses, tying the real world  to the information space that surrounds us in our everyday lives. This means phones, PCs, and the like will require the kinds of rich, real-time interactive 3D interfaces that right now only WebGL can offer in a cross-platform, stable, browser-based way.

There is clearly only one direction forward for Microsoft and 3D on the web.

WebGL is the way.

 

 

  1. #1 by Mary Branscombe on June 17, 2011 - 2:34 pm

    so you don’t think the shared graphics memory screen-scraping attack is an issue? if that WebGL game I’m running while I log into my bank account just happens to scrape the login screen for my user name, hey, whatever….

    • #2 by avi on June 18, 2011 - 9:29 am

      Mary, the WebGL spec does not allow scraping your screen, even in shared memory situations. You can’t ask for texture IDs or memory you don’t own. It could be used for phishing attacks in the exact same way HTML+CSS or Canvas can — rendering fake content and asking for passwords. That’s true of any rendering technology in the browser.

      On the contrary, we would like to see the ADDED ability to render HTML+CSS content to a texture for 3D, but that should be limited to DIVs owned by the same context and domain (e.g., an off-screen DIV) and not cross-domain. Perhaps even zeroing pixels (alpha too) that come cross-domain (ads, images, etc..), or simply clipping to the owned box, whatever shape.

      The only browser afaik that has the ability to grab arbitrary screen regions to texture is Firefox, and they intended to restrict that feature to chrome extensions only (edit: there was obviously a bug). Restricting it to same-domain DIVs would be better IMO.

  2. #3 by Kevin Gadd on June 17, 2011 - 11:04 pm

    “Somehow we survived the existential threat of native code plugins taking over our PCs.”

    This, I think, is little comfort to all the people who suffered identity theft or other significant inconveniences as a result of the security threats we’ve seen afflict the web so far. It’s fair to call MS’s position fear-mongering, but the position expressed in the ‘Considered Harmful’ post is soundly rooted in reality and it describes security risks that have not yet been fully addressed by any WebGL implementation. This is a Big Deal, and Microsoft is not the only company nervous about it (Notice how no iOS device exposes WebGL support to the public, despite Apple being on the committee).

    It is important to attempt to advance the web platform as quickly as possible, by making technologies like WebGL available to early adopters. But it’s also extremely important to do this in a cautious manner and do so in a way that allows for quick and complete responses to security threats before they cause harm. I am not yet convinced that any browser vendor is equipped to do this, so the fact that WebGL is now enabled by default in multiple browsers makes me particularly nervous.

  3. #4 by sulfide on June 18, 2011 - 5:48 am

    ahh now along with noscript i’ll be running nogl too ;o

    • #5 by avi on June 18, 2011 - 9:35 am

      Sulfide, do what you need to to feel safe. But my expectation in the near-term is that WebGL will not be enabled globally by default, at least not for every website. Just as mobile browsers ask if an app can use your camera and location, I think WebGL will soon require explicit approval to run per site per user, perhaps with a global whitelist of sites or trust certificates and/or signed shaders.

      The main thing we need to get to is 3D web apps. In fact, if there are WebGL-enabled ads, I expect those will be included _inside_ 3D web apps, not around the edges anyway. So I’d be fine if we had a way to disable WebGL for all but the same-domain (as the main URL) site.

  4. #6 by avi on June 18, 2011 - 9:40 am

    Kevin, I suffered identity theft. I had a fake facebook page created with my picture and my co-workers apparently phished (in my name).

    This has nothing to do with WebGL. The only exploit I’m aware of is that a degenerate WebGL shader can lock or crash your machine on some HW and OSes. And that can be fixed with better drivers and/or watchdog timers.

    The risk of identity theft comes primarily from poor containment in DOMs. In my opinion, things like browser ads (today) should be completely sandboxed and virtualized such that they can never get to your information through the DOM. With the lack of accountability for ads coming through multiple levels of brokers, this is rife for abuse. But again, this has nothing to do with WebGL — it’s a rendering pipeline, not a new DOM.

    Also, Apple can speak for themselves, but I wonder if disabling WebGL on Safari (mobile) is more about the native vs. browser debate.

  5. #7 by Benoit Jacob on June 18, 2011 - 11:46 am

    Avi,
    Thanks for the very interesting and encouraging blog post. (From a Mozilla WebGL implementation developer).

  6. #8 by Benoit Jacob on June 18, 2011 - 11:52 am

    @ Mary: assuming that by “the shared graphics memory screen-scraping attack” you’re referring to http://blog.mozilla.com/security/2011/06/16/webgl-graphics-memory-stealing-issue/

    This vulnerability is entirely specific to the WebGL implementation in Firefox 4, has been in fixed on May 26 in the Mozilla codebase, and Firefox 5 has the fix (coming out in 3 days). I regret that the security researchers who discovered it didn’t follow the usual practice of letting us release the fix to users before disclosing the vulnerability.

    In any case there’s nothing here that can be used as an argument against WebGL in general.

  7. #9 by Benoit Jacob on June 18, 2011 - 12:02 pm

    > my expectation in the near-term is that WebGL will not be enabled globally by default, at least not for every website.

    WebGL is already enabled by default for all websites in Firefox 4+ and in Chrome (I think 9+). User confirmation, in this case, is not something we currently feel comfortable relying on for security: most users are not technically able to make this decision, and also many people just don’t read dialog boxes, so our job is to make WebGL secure without user interaction.

    • #10 by avi on June 18, 2011 - 1:31 pm

      In this case, “globally” could mean either in all browsers, or for all websites. :)

      Yes. The whole reason for not being a plugin is to be seamless and ubiquitous. Agreed. However, given shaders are akin to native code, I still expect to see some pullback from that goal in the short-term. I’d be happy to see solutions that can block potentially unsafe code without user interaction.

      I also think plain old driver quality issues (which Angle can help fix, ironically) could nudge things in the “selective approval” direction, if WebGL features are present but just don’t work well enough for some apps or sites due to crappy drivers.

  8. #11 by Rafal Lewczuk on June 20, 2011 - 2:34 am

    Let’s face it – it’s politics, not technology. 3D gaming is a strong selling point for Windows. DirectX is Microsoft’s proprietary standard and huge number of 3D apps are locked into it. They will fight tooth and nail to keep this status quo – and if FUD won’t help, patent lawyers will.

  9. #12 by avi on June 20, 2011 - 9:14 am

    @rafal, I completely disagree. You know why? WebGL will likely be implemented on DirectX on Windows even by 3rd parties.

    That’s an artifact of DX having better drivers and OS support, meaning it’s safer, more consistent, and more reliable on Windows at this point in time. Even the open source Angle project is using DX where it exists.

    I cut my teeth on OpenGL and still have a fondness for it, but WebGL is all about browser-based 3D, of which there is no DX alternative. As I said, WebGL is the only way.

  10. #13 by Matthew Raymond on June 20, 2011 - 1:00 pm

    @Kevin, Apple is testing out using WebGL in iAd for iOS 5. Presumably, they could enable WebGL for all Web content at a later date.

    @Sulfide, WebGL requires Javascript, so you can disable WebGL simply by blocking the execution of any scripts on the page, which I believe is what NoScript does in the first place.

    @Avi: “I mean, the exact same graphics hardware that WebGL uses is available to native applications running DirectX or OpenGL on your PC and/or phone. Are we going to ban downloaded games because they might, in some universe of possibilities, harm our computer or cause us to, God forbid, reboot?”

    I’ve been wondering the same thing. In fact, if WebGL scrubs the shaders so that they are limited to the capabilities of OpenGL ES 2.0, then wouldn’t native apps have access to shaders and features that are even more powerful?!? What happens when we want to used WebCL or OpenCL to take advantage of shader units on Fusion-style processors? (Example: WebCL version of the Bullet Physics Library.)

  11. #14 by Shmerl on June 20, 2011 - 2:53 pm

    Thanks for standing up for WebGL from the Microsoft side. But I still find it hard to beleive, that Microsoft would put their usual attitude aside, given their treating of OpenGL in general.

  12. #15 by James on June 20, 2011 - 11:34 pm

    It seems a bit hard for me to believe that DX vs. GL doesn’t impact the decision-making. To have to keep a translation layer competitors don’t have to (if WebGL takes off GL drivers on Windows will get better) or choose to embrace GL in Windows despite the DX investments over years seem like very unappealing choices.

    I wonder how much of it is influenced by the naming. An interesting backstory: The canvas tag enthusiastically endorsed by IE9 is a Javascript reflection of Apple’s CoreGraphics API. It was designed for Mac Dashboard widgets and accidentally exposed to the web; but the web really needed a 2D graphics API and it was a good one. (Most of the odd behaviors of canvas are CG bugs that got baked into the spec – especially around shadows where CG was really wierd). And MS implemented it on top of Direct2D. I think this debate would be different if WebGL had kept its original name of Canvas3D instead of the Khronos marketing guys wanting to get ‘GL’ in the name, since it would seem like a natural extension instead of something very different.

  13. #16 by przemo_li on June 21, 2011 - 10:30 am

    MS have its own Silverlight 5 XNA3D. So no way WebGL will be implemented first.

    But as XNA3D faces the same problems, MS will fix its own OS 3d graphic subsystem. Its only matter of time.

  14. #17 by przemo_li on June 21, 2011 - 10:34 am

    Oh, and MS folks did not realized what WebGL is from beginning or tried very hard to downplay its meaning from the beginning. Eg. by comparing WebGL to acceleration that is already in IE9. Or by stating that WebGL is Mozilla only thing.

  15. #19 by Sakamoto on June 22, 2011 - 7:39 am

    When I first read the news regarding MS shying away from WebGL my thoughts were similar to yours… just fix the problems.. However, there maybe another reason for MS to stall… WebGL will allow a user experience closer to what Windows itself offers. Reducing long term demand for the OS.

  16. #20 by Brian on June 22, 2011 - 1:35 pm

    [Avi’s response in-line in brackets…]

    Yes Microsoft is right – Worst it can do is crash your machine – Just what world do you live in!

    [ read the very next words, brainiac. “We can do better.” ]

    That is horrific – yes maybe Microsoft should work on in. But it’s irresponsible to build it into the browser, Plug in’s you have a choice.

    [Well, there’s the rub. Early ActiveX controls did self-install, sometimes via exploits. Now, with better protections, you do have a choice. That’s my point. Security = good. ]

    Built in you don’t, which means security aware companies have to ban browsers with it built in – which is a pain!

    [People who ban browsers for having wanted features will find themselves obsoleted.]

  17. #21 by David on June 22, 2011 - 4:54 pm

    Good to see plenty of different viewpoints to the Microsoft response to WebGL. I’m merely happy to see evidence Microsoft does at least consider security issues these days before launching new features on an unsuspecting customer base. There has been considerable historical criticism of Microsoft’s attitude toward foisting vulnerable code on customers in the past, and moving to fix problems only after something bad happened. Cheers to evidence of a change in attitude.

    And this much angst over 3D in browsers! Playing games is the only serious issue with 3D. So download, or heaven forbid purchase a DVD, and INSTALL your game separately to the browser. It will perform better anyway…

  18. #22 by Sidharth on June 25, 2011 - 11:43 am

    Lovely, passionate article.

    As others have pointed out around the net, Silverlight 3D is just as risky as WebGL.

    Just look at http://bodybrowser.googlelabs.com/

    Anybody who has see this and does not think that WebGL is the future is kidding.

    Now go and plug all security holes..Microsoft. Stop spreading FUD.

  19. #23 by Lee on June 30, 2011 - 10:15 am

    It’s actually more disturbing to me to hear that this is not a play against WebGL as a platform.

    It’s disturbing because of the prospect that such a large force in our industry could let overly conservative thinking trump innovation. Maybe conservatism is a valid philosophy for SQL Server, but not in the war to stay relevant in browser platforms.

    The people I know who work at Microsoft, and apparently Avi Bar-Zeev who I do not know, would not make this one size fits all strategy mistake.

  20. #24 by eddy on November 12, 2011 - 6:02 am

    This is just because of conspiracy theory of Micro$oft.

    WebGL is based on OpenGL and OpenGL is the bigest rival of Microsoft’s Direct3D (part of DirectX). OpenGL is Industry Standared.

    If people start using WebGL and OpenGL then M$ is going to loose big money. This is the main reason why M$ doesn’t want to integrate WegGL into its browsers.

    If webGL is based on Microsoft Direct3D no wounder M$ would keep quiet and support it.

    This is the only reason why a lot of people are forced to hate microsoft.

  21. #26 by przemo_li on December 12, 2011 - 3:16 am

    You need to rewrite your post now, when SL5 XNA do the same thing as WebGL!

    Now it is DX vs OpenGL. Now it is hypocrisy (they ship insecure feature or they laied about unfix-ability of WebGL).

  22. #27 by Leila on September 28, 2012 - 4:46 am

    @przemo_li,

    The main consumption for Silverlight5 is Windows desktop, RT and windows phone space. Silverlight on Web is not even supported in IE10-metro browser in Windows 8! So Microsoft is NOT promoting the Silverlight5 for web. However, they do support Flash natively on IE10-metro…

    The difference between Silverlight5 and WebGL is that they have the ability to blacklist and white-list the graphics card drivers (especially the latest one from Radeon and Nvidia) and its on user’s discretion to “allow” the access to the untrusted driver in Silverlight 5. In open web, WebGL doesn’t guarantee such protection as of now.

    The right (and assertive) question should: “Is the claim legit?”. Unfortunately, it is. Sooner the web standards for WebGL will revise the standard recommendations and come up with agreeable terms from all parties (security conscious, openness-crazy..), the sooner will Microsoft implement WebGL in Internet Explorer.

    WebGL can (and is) getting inspirations from the said SL5 and Adobe Flash implementation of protection, which is the right thing to do. Flash has already stated that about 80% of web-graphics are inspired by Adobe flash technology. But they also admitted that its time to move on with new-open web standards. So did Microsoft.

    Finally, if you concentrate on the transition from IE8 to IE9 and then to IE10, you will notice that IE guys are serious in supporting handsome amount of HTML5, CSS3, SVG standards. It makes me pretty confident that they will introduce WebGL to IE soon!

    Like Microsoft employee admitted, WebGL is the future. At the same time it needs to get mature from security point of view before jumping to the implementation.

(will not be published)