Monday, November 22, 2004


An appropriate link for a traveling day. As more computer interfaces begin to bring usefully indexed and more massive amounts of data, we're really going to see the rubber hit the road in terms of large scale integrated analysis. Expect efficient extraction of significant features from large datasets(like maps, statistics, advertising data, financial records, text databases, traffic data) to be increasingly significant to business. The rise of researchers and supercomputing for hire will only increase as organizations begin to realize how much value they're drowning in, but can't realistically get at without searching, interface, and filtering.

Google is a very visible, very public front-runner here, but most of their services are very unsophisticated. Other firms, like Oracle, Microsoft, IBM, and CAS in many cases have much deeper search problems, and much more established revenue streams. Google is certainly increasingly dangerous to these markets, and is aggressively expanding their dataset, likely betting that a crosslinked, massively indexed series of search services will be more valuable than a more deeply searched, or better annotated one.

It's perhaps characteristic of Computer Scientists to overgeneralize, and for Artificial Intelligence researchers to do so in a manner bordering on the ridiculous. The search problem certainly seems like a weaker version of the perceptual issues in cognitive science to me, though. I anticipate that these fields will, at the highest levels be interlinked in theory, if not in practice.

I'm going to Salt Lake City, to visit with friends, and connect with family for a short time. I'm looking forward to it.

I'll be back the 30th. Please behave yourselves in the meantime.

Some status updates, I managed to finish the integration framework for all our virtual world testing for the next few steps. This is basically the system we'll use to transition from disconnected cognitive testing to a continous, task switching, big brain kind of model.

Whereas before we were testing specific ability in repeatable tests, and establishing the perceptive categorization, the learning algorithms, the exploration routines in separate domains, we're now upping the complexity and the richness, by jamming all these tasks into a big messy place, more like the real world.

There are several problems with this, the first being the difficulty of testing, which we've actually got a model to reduce this, by isolating particular tasks at particular moments, and generating baselines to compare against on the fly. It will really be interesting over the next month as we get our first experience with how all these capabilities interact in an unplanned fashion. I am cautiously optimistic. Many of the cognitive algorithms we have would operate better in a more data rich, and varied environment than the rather sparse controlled situations we've been using up to now.

a key point in cognitive design is that brains are supersystems, and it's very artificial to imagine that each module would be able to operate on it's own. Seperating out for testing and development is fine, but the capabilities and sections of intelligence must exist in an expanding ecology of goals and functionality, or the whole things falls apart very quickly, because of the complexity of the interactions.

Getting to this point before my vacation was a purposeful thing, as during the trip I plan on thinking through the more theoretical issues that face us now that we are in this expansion, scaling, and integration phase.

We're going to be dealing with much bigger brains, far more mental objects, and many many more patterns and actions to be correlated. All good things, but their effect on the system, and the development they will push us into must all be considered, lest we waste time. Ideally of course, we would push in all directions at once, perfectly co-ordinated, with no duplication of effort, but we dont' have either the manpower or the telepathy to do that. So we need to proceed piecewise, with some overall prospectus.

pity the poor man who is not confused by his job and his tasks. for he is either blind, or miscast in his role. and for either, he is learning nothing.

No comments: