Data Miner 49er

If you have a contest, they will compete.

A few months ago, I started participating in Yahoo Answers, somewhat off and on. I realized there was only one topic I knew well enough to answer questions: tennis. Most of the time I go there, I find the dumbest questions ever. In classes, they tell you there's no such thing as a dumb question, but it seems like there is. And plenty of it.

But what causes people to answer such insipid questions? The thought that they may gain points and move up the ranks of Yahoo Answers gurus. I thought I was decent at it, but was I answering every day? No. Did I have expertise in more than one topic? No. So there was bound to be someone who knew a lot more.

Wikipedia, that grand experiment of social networking creating content mimicking encyclopedias, is mostly written by a small group. That is, something like 1% writes more than half the content. I may not have the numbers right, but that's about a ballpark figure.

How important is information? With so much of it proliferating around the world, there's starting to be people with an odd skill. Using a combination of RSS feeds and search engines, there are those who have the ability to find "interesting" articles. Flickr, the Web 2.0 poster boy photo site, also ascribes interestingness, a Bush-like phrase if ever there was one, to photos, but more than likely based on how many views it gets, and how many people tag it as favorite. Flickr doesn't reveal the algorithm used to pick interesting photos so people don't try to game it.

However, social news sites like reddit and digg, much like Wikipedia, rely on a handful of people who are obsessive-compulsive about finding interesting articles. Since anyone can do it, even a teen can get into it, and get a bunch of "karma", the currency of goodness at these social sites, the gaming aspect that makes people want to compete in finding articles that people care about.

The reddit community, once a geekish set that looked to interesting topics on programming, has become very American left-leaning, anti-war, anti-Bush. Of course, there's the occasional interesting photo, the transgressions of authority (police harassing "innocent" folks are typically big news), and even a few geeky articles here and there. Sports columns don't really make it to the top.

Does this kind of skill mean anything in the information age? The skill of being a human data miner, finding articles of interest, and having others vote on its relevance? People do it for free now, but would someone pay? To some extent, it already happens as Netscape wanted some diggers to work for them, but is it enough of an industry to hire thousands or millions or is such data acquisition the purview of the very few for the entertainment of the geekish many?

We move into a new era of "information", and the question is what information jobs will look like in the years to come.