More Detail on Skyline v. Google


Google Copyright Blog: Google Prevails in Earth Patent Dispute

FYI, the Google Copyright Blog, linked above, contains links to the parties recent motions and the judge’s final ruling in the case. I just read through the judge’s decision and it’s fascinating, though I’m not sure how to explain why without getting into much tech talk and approaching the limits of my old NDAs. However, since the ruling is now public, I can comment on things that are public.

Suffice it to say, one of the critical issues the judge determined was that Skyline has in no way proved that GE uses a "renderer" as the patent describes. Now, anything that seems to "render" 3D images can be said, in some broad sense, to have a "renderer" somewhere in the code. But it’s not that simple. An apparatus that "makes coffee," to use the judge’s example, is not necessarily "a coffeemaker."

It seems that here, the critical difference is that Skyline’s "renderer," as they described it in that fuzzy curious patent of theirs, includes both the image generation routines and the data streaming functionality, among other things. The judge ruled that there is no common block of code in GE that does the same thing. It sounds like a bit of a technicality at first. But that clear design decision from early on is one of the things that probably makes GE more efficient and more responsive than Skyline’s approach.

As I understand Skyline’s technical history, they started as an optimized software-only "height-field" terrain renderer, way before cheap 3D hardware. That approach is practically obsolete nowadays, given modern hardware acceleration, except in special cases like per-pixel "bump mapping" for surface detail (aka "parallax bump mapping", done in programmable graphics hardware). And that might help explain their stated approach to so tightly integrate data-streaming and drawing.

But GE comes from a real-time vis-sim background, where things work very differently. The high(est) priority is always to maintain a solid (often 60hz) frame-rate, which means decoupling almost everything from the pure act of drawing polygons as fast and as deterministically as possible. The streaming parts of GE can be most easily thought of as a whole other program (a thread at least), running asynchronously from the part that draws the Earth. Skyline’s patent claims fail on those grounds alone.

Another interesting aspect of the ruling, and a whole other ground for dismissal, is the idea that GE doesn’t send "coordinates" as described the patent. That jumped out at me when I first read the patent, when the suit was announced so many years ago. The patent is very specific that the "renderer" must send coordinates and get the corresponding "pluralities" of terrain tiles and whatnot. Again, that makes some sense for a height-field renderer, at least one that loads new detail from fast memory. It’s one of the reasons I reasoned that they might not have solved these problems on the grandest scale. It doesn’t work too well on the internet.

Without going into too much gory detail, it’s incredibly inefficient for a whole-earth-streaming client to send hundreds of thousands of similar messages to a server, each saying, "hey, I need another tile for this X,Y and this level of zoom." The judge seems to understand that GE has a much more efficient way of both encoding space and of telling the server what it needs or may soon need. Again, there is an efficient decoupling of rendering-side concepts, like "culling" invisible parts of a scene away, vs. the hard-to-scale aspects of streaming terabytes of data over 100kbs internet connections.

And the judge seemed to hone right in on this, as well as the sheer stupidity of Skyline reporting their algorithm as fetching new tiles when the resolution is "not equal" to the desired resolution. You really only want new tiles when the resolution of the current tile set is too low (a greater-than test vs. a not-equals test).

I’m quite impressed with the judge’s understanding of those aspects, though not having heard the actual arguments and expert testimony, I don’t know how much credit to give to Google’s lawyers as well. And while the judge seemed to avoid ruling on the actual validity of the patent, that last bit, I hope, would make it easier for anyone else to challenge the patent. The invention, as described, and perhaps too-hastily blessed by the USPTO, seems not to work as well as is claimed.

Anyway, what it comes down to for me is I’ve regained some sense of hope that the patent system occasionally works. Of course, it requires the resources of a GooglePlex to actually fight and win against a bogus patent claim and get to the point where justice (IMO) is done. A smaller company might be forced to settle in the same situation. But I consider this a small victory for fairness and innovation. And I send a hearty congratulations to the GE team for dealing with and overcoming this unfortunate obstacle.

[Afterthought: it looks as if discovery in this case involved disclosing GE's full source code to Skyline. I wonder what happens if Skyline appears to use that information to improve their products? Does anyone know?]

Here’s the text of the decision.

  1. #1 by shiv reddy on March 22, 2007 - 9:35 am

    i read a detailed report in the press of the judge’s ruling in this case, but found no explanation as to why the judge did not consider that the alleged steps employed by Google Earth to perform the rendering did not qualify as equivalent to the ‘renderer’ in the asserted patent under the doctrine of equivalents. so i thought i would hit the blogosphere with this question hoping someone would have an answer. would you happen to know? thanks.

  2. #2 by avi on March 22, 2007 - 10:01 am

    Shiv, if you have a link to that detailed press report, I’d love to see it.

    I don’t completely understand the law on what counts as equivalent. But the judge seemed to take the wise (IMO) opinion that simply because two things seem to produce similar outputs, their workings (i.e., what is protected) may not at all be the same. He cited the relevant case law in the ruling, FYI.

    Patents should IMO be limited to what is described, not some general class of inventions that might follow, for which the "inventor" has no reasonable claim to have invented — and perhaps can’t even describe except in general terms. [And then there are patents granted that don't even do what their own inventors claim. There's a Sony mind-reading patent like that. They guessed they could use sound waves, but it's not clear they even tried to build the thing to see if it worked.]

    So if I patent the incandescent light bulb, I can’t claim the rights to candles, florescents and LEDs, unless they use the method I described (which they don’t). Google Earth doesn’t use the same method as skyline, except in the most generic sense — it gets user input, it transfers data over the internet, it draws imagery. But it doesn’t do that in a way that reasonably resembles Skyline’s method. The fact that Skyline itself apparently couldn’t identify a single part of the GE code that corresponded to the "renderer" they described is telling.

    Anyway, I realize that some recent "inventions" (like 1-click ordering) would be hard to distinguish if they visibly seem to do the same thing. It seems to be the user experience, not the underlying method, that is patented in those cases — I think those patents should not be granted anyway for that very reason, among several others I could name.

    So if Skyline had patented "a 3D interactive globe with terrain, imagery, and other geographically-fused data streamed via the internet," they might have had a better shot claiming GE was equivalent (though it would be hard to claim they were first). But then again, such a patent should never be granted in the first place. And, as it turns out, that’s not the patent Skyline wrote.

  3. #3 by shiv reddy on March 22, 2007 - 5:44 pm

    Thank you for your response, Avi. I would post the link to the article here, but it was sent to me in a Bloomberg IP Law Report licensed for internal circulation at my firm.

    The doctrine of equivalents allows the federal court system to hold an alleged infringer liable even though there is no literal infringement, if it can be shown that there is atleast equivalence on the missing element. So if the patent in suit claims elements a, b, and c, as either steps of a method or elements of a system or an apparatus, and if a and b are shown to exist literally, but only an equivalent found for b, under the doctrine, the alleged infringer will be found liable.

    from my understanding of the judge’s findings, there was a component in GE that performed the “rendering” steps of skyline. my confusion is about what stopped the judge from going further with that analysis to say that there was atleast equivalence under the doctrine.

    i dont know about the other elements in the patent’s claim, but since the discussion in the article was about this “renderer,” i am assuming that the patent is so broad that it covers all the “user experience” aspects, generaly sweeping over the rendering aspects; and the judge found that since Skyline’s patent disclosed merely the “rendering” steps and not an actual “renderer” component, there was no literal infringement. i understand from your response, post and other material, that there are many other inherent differences; but if the judge could find that GE had a renderer and Skyline had steps that disclosing its general functions, what kept him from drawing a conclusion that there was equivalence.

    i also had a question; seeing as how you were involved in GE’s design.(?) so perhaps you know. did Google turn over its source code for GE to skyline’s lawyers so they could perform an infringement analysis?

  4. #4 by avi on March 22, 2007 - 6:23 pm

    As to Google’s source code, it certainly sounds like it was made available. The judge wrote something to the effect that Skyline hadn’t pointed to specific areas of the code, etc… But I don’t know first hand.

    My reading of the decision is that Skyline described a specific component they called a "renderer." Their description implies that their "renderer" is responsible for a lot more than coloring pixels on the screen. Again, I’m no lawyer, but I think any equivalence between the two products is very superficial.

    I would certainly hope that any court would be very careful in trying to equate things. In the case of some widget, if I use one commodity part in my invention and someone else uses another commodity part in theirs for similar purposes, that might be reasonably called equivalent, though the parts are clearly not the same. But if my novel and custom-built part is a core claim of my invention, I would hope that there would be much more scrutiny as to whether they’re really the same or whether the similarities are superficial.

    I think that goes back to the example of light bulbs vs. LED lights. They might be considered equivalent in a patent dispute over some novel parabolic mirror for a flashlight that reflects more light from a given luminance. But if the question for the court is about whether LED lights infringe on incandescent bulbs, then we can’t simply look at the fact that they both produce light. How they work and how they’re composed becomes critical. And I often wonder if some of the problems we have in the IP area are due to people abusing this notion of equivalence (not that this is the only concern I have).

    Anyway, I don’t believe the Skyline patent covers the user experience, or even mentions a 3D globe iirc. It seems to try to lay claim to inventing a method for sending hierarchical blocks of terrain data over the internet, which in other areas is often called "a multi-level cache" or "client/server" and would be an obvious solution to any skilled person in the trade. Several of the methods GE uses might not be obvious, but we chose to protect those as trade secrets, which is why I asked about how Skyline could use the source code if they have it.

    But if you’re a lawyer, I’d suggest reading the decision directly. You may be in a better position to interpret it. It’s linked at the bottom of my post. Let me know what you think.

(will not be published)