The FBI's Secret Scrutiny

this is some very good reporting by the Washington Post. Library and Commercial databases being accessed without oversight is a serious problem. You better believe that Google has been served these, whether they fought it or not is hard to say. The only reason I'm not loading up on tinfoil and proxy anonymizers is the fact that no one is really equipped to actually walk a database that big in any reasonable time-frame for any but the most superficial or narrowly defined reasons.

Of course, this is subject to change. And the NSA still has one of the largest communities of mathematicians and computer scientists outside the oversight of.. well, anyone, really. It's not inconcievable that they've spent a good amount of time working on software and hardware tools for sorting such information well.

As something of an enthusiast for meritocratic processes, like capitalist competition and open, iterated collaboration, I tend to believe that they will not progress as quickly as more public organizations. But they gain in their focus. Cryptographers know that the NSA is ahead of the public, because they have the narrowness and numbers to stay ahead. That will be true for some other aims of the NSA as well. In addition, they are not necessarily bound by copyright and patent restrictions. So you can bet they use the best software they can find, buy, invent themselves, or simply steal. In that sense, they take advantage of the best parts of public discourse, and contribute little.

The government organizations may lag behind state of the art in architecture, paradigm, staffing and methodology, but I imagine they are aware of those shortcomings. It's sad when the best we have to look foward to is the incompetence of our Ministry of Justice.