It’s not often since I started at Microsoft 3 years ago that I get to blog about what I’m working on. It’s not that anyone had ever explicitly told me not to, but sharing my daily web research and observations publicly (even if the source links were already public) would have likely compromised the secrecy I’d agreed to, since I tend to blog about whatever’s most on my mind. [And forget twitter. My tl;drs alone are more than 140 characters]

With my current project, the situation is much improved blog-wise. We concluded recently that the best way to accomplish our goals was to be open and even solicit cooperation from outside. Quite a concept!

The project’s name is Read/Write World.

If you haven’t heard about it, here’s Blaise’s (my boss’) recent keynote at this year’s Where 2.0 conference on Earth Day:

Below is a link to a live demo we recorded last week of something I’ve spent the prior week or so getting barely polished enough to show.

It’s a real-time editor for the Read/Write World. What you’re seeing is me flying around the map, sifting and editing some data by moving it around in real-time, incorporating (shown with the red lines) real-time match results from a great new service that better connects bits of data together (photos in this case) by visual similarity, aka feature matching.

Behind this visual front-end lies the unseen but much heavier effort of the last six months — real-time read/write Azure services ready to handle ungodly amounts of information. Yes, real-time, as kind of like a MMO, except we’re not tracking players but rather changes in the world itself. Building that, and a new (in some ways old) language called RML, to help describe that information, has taken up most of my time.

Our first goal is really straightforward, but not necessarily easy. Microsoft has lots of data in lots of formats. We also have to cater to every browser, every mobile device and platform known to Man. Multiply those together and you need a thousand talented people to just to keep parity with the rest of the world. What we needed was a way to make one application use one data format do more with less effort. In mathematical terms, 20×10 = 200 but 1×1 = 1.

Good in theory, hard to do in practice. But if we succeeded, the benefits would accrue not just to us, but across the board.

So on the “one application” front, we’ve focused on HTML5+CSS3[D]+WebGL as the most promising way forward (some Silverlight too, just since you wondered). On the data front, we’re building this crazy simple metadata format called RML. Here’s my quick writeup on “What is RML?” if you’re interested.

tl;dr: it’s just basic JSON structures with enough reflection to make it so we can transform to/from other formats/versions very easily, and write butt simple webapps that use it and render it to boot. At the meta level, it’s an Entity-Relationship model woven into a world-scale scenegraph.

Now, one of the debates we had was, does it need to be a standard? Our conclusion, which I’ll elaborate more on in my talk at the ARE2011 conference next month, is “no.” We’re not trying to change anyone else’s standard, nor do we require (or expect) that other people adopt our format as their own. We’re even expecting our own users will grow and change RML organically to suit their needs, without necessarily asking permission. We’re trying to make that easy, supported, and robust. A lot of this is inspired by Open Street Map, btw, which has very compatible goals.

Our intention is to gradually make public some beefy servers that can read and write this RML stuff, mostly visual data for now, such that anyone can easily store and retrieve their geospatial metadata here, for free, and even write apps that do useful things with it, for free.

Caveat: we fully respect copyright and claim no implicit or explicit ownership over other people’s data. So while our service might be free, some of the data we refer to might still reserve rights. For example, the images might be hosted elsewhere and carry a copyright tag. We will respect and pass-on those assertions to any consumer of the data, of course, but still strongly encourage content owners to choose creative commons wherever possible). Beyond that, our big “value-prop” is that we’ll crawl the data we store, trying to find new ways to connect it so it’s more useful than just a list of random cruft sorted by lat/long. That’s one of the hardest and most unique parts of the system.

So, to cut this off before it becomes a novel, between this site and the official ReadWriteWorld.net, I and others on my team will try to keep things pretty current and informative. The main difference is as always that RealityPrime contains no official Microsoft spokesmanship but merely my own personal opinions, which carry no warranty and can change without notice.

Last caveat: I might not be able to answer the most likely questions just yet, especially about availability, betas, and so on. That’s still being worked out. I’ll even be a bit hazy on the specifics of our functional RML format right now, mainly because it’s undergoing rapid evolution as we throw more and different collaborators into the mix. But any questions I can’t answer will go into a queue for the official FAQ.

Change, in this case, is good.