Aardvark, Part 2

The documentary about the creation of CoPilot was not written in two parts. In fact, I think the whole thing lasts all of about an hour. Even so, I watched it in two parts, because I watched half of it at 2 AM, and the blogged for an hour or so, and then went (mostly) to sleep.

This morning, I woke up, and went to watch the second half.

The second half is, actually, a bit better than the first half. Again, much like the first half, there isn't much said about the actual creation of software. What makes these kids so good at software? How did Fog Creek organize this project with some assurance that the solution would actually be feasible in twelve weeks? What was the software development process like? How was the architecture designed? How long did it take to get up to speed on the work environment?

Clearly, such things would not have made for good watching, and yet, it's exactly the kind of thing I care about.

About ten minutes into the second half, the film makes a wild veer, and starts talking about Paul Graham. For a while, I only knew about Paul Graham through a book he wrote on Lisp. Lisp is acknowledged as the second oldest programming language, behind Fortran. I should say "surviving language", as I'm sure there were many "languages" that only a handful of people ever used and therefore don't qualify as languages.

I was then told about his book, Hackers and Painters, where he discusses what it means to be a hacker. Like the documentary, I'll make a divergence as well.

In the 80s, when PCs became ubiquitous, the term hacker grew to mean a person who broke into computer systems illegally and, on the benign side, left some indication they were there, but otherwise did no harm, and on the malicious side, tried to bring the system down, or stole secret information, or altered information.

It wasn't until later that hacker came to mean a person who was really good at coding, and a hack came to mean a clever way to code something up.

It is in this sense that Paul refers to "hacker". He notes that only a small percentage of people in computer science can be considered hackers. Most computer science majors know that, while the professors they work for are bright, and possibly even brilliant, many are average coders, and some, oddly enough, don't even code. They are theoreticians, or they pass the buck onto grad students to deal with the painful aspects of programming. Those few who love to code claim there's nothing better, and yet, to code well means you're constantly learning how to make the best of the programming tools out there, and to learn new and better tools.

Many bright computer science types simply find that keeping up with coding trends is futile. It's a waste of their brain power to learn the newest fad. Why learn Ruby? Why learn Java? Why learn C++? There are many a CS prof. whose last programming language was Pascal, and they're happy they haven't bothered to learn any new programming language in twenty years. It's like learning a new foreign language every year. What's the point?

But hackers love programming languages, and not just for the sake of learning something new. In fact, Paul Graham's favorite language is Lisp, which is an old, old language (though it looks far different now, than it did when it was invented). He says it allows him to write "hacks" more efficiently than any other language.

Paul Graham isn't just an academic who talks about code, either. He and Robert Morris, he of the famous Internet worm of the late 80s, went to write software that eventually got sold to Yahoo and formed the basis of Yahoo Stores.

Graham, like Spolsky, also believes in hiring the best. The rest simply aren't worth it. They believe in that elite hackers should rule the world, or at least, the software industry, and by extension, that hackers are born, not made. Or, perhaps more accurately, you can make a very good hacker great, but you can't do anything useful with an average coder. It would take weeks of constant hand holding to make the average coder decent, and even then, they'd always need more hand holding. A top notch coder gets things quickly, and then figures out new ways to do things that you never thought of.

Graham's been criticized for this belief. In particular, Graham has esposed that hackers should only be given cool stuff to work on, that the tedious boring stuff should be left to the petty programmers of the world. It is for this reason, that some argue that hackers are ill-suited to a startup company, because someone has to write there boring pieces of code, and if the hackers lift their noses to the air at such tasks, a startup would fail.

Graham has his own company called Y Combinator, which is about as geeky a name that you could ever come up with. I suspect the first question he asks potential employees is "Explain what a Y combinator is". Those interested in lambda calculus would know that the Y combinator is a way to implement recursion in lambda calculus.

Aardvark'd discusses this company for about ten minutes, and it becomes like a recent episode of Lost. OK, let's talk about Lost. Lost is ABC's big hit about a plane that crash lands on an island after leaving Sydney, Australia, ostensibly to head to California. This island appears to have invisible monsters, and secret capsules, and so forth. Essentially, it's about how the passengers try to figure out how to get off this island, and the people they encounter, including "the others".

Anyway, in the pilot, you learn that the plane broke into two parts, with the main part falling on one part of the island. It was assumed the other part went into the sea, and everyone died. Only, they didn't. One episode was devoted to telling what happened to the people in the tail section. This covered over a month's worth of events in 50 minutes.

Ten minutes of the documentary are devoted to Y Combinator. You get to see Paul Graham, and some of the kids that go work for his company. Instead of being called employees, they appear to be called "grant winners". However, there's so little time devoted to this that you never get to see a compare/contrast between the way these two companies work. Just as it appears you're going to see a new cast of characters, like Real World 2 meets Real World 1, it goes back to Fog Creek.

One thing you realize about documentaries is how they manage to avoid people getting self-concious about the camera. The kids are always suppressing a laugh, and seem to feel awkward making the documentary. One scene involves the recounting and reenactment of a large cockroach in the Fog Creek lavatory. This involves a cockroach eye view of the two interns who proceed to kill the cockroach.

Ever listen to NPR segments. Unlike television segments, there's no video in an NPR segment. It's radio. It's audio. They rely on sound to tell their stories. In particular, music can be a big part of the story, sometimes serving as counterpoint to the story. For example, I remember one angry story of how Bush's administration completely flaunt the rights of Iraqi prisoners, due to their wanton ignorance of the law. As angry as the subject matter and words are, the voices of the commentator and the music especially, are muted, if not, upbeat.

There are several songs in the film, and they're so obviously keyed into the documentary itself, that after a while, you realize it had to be written for the documentary. And so it was. (I should really call these things infomercials, because that's more or less what it is). Lerone Wilson not only made the documentary, he also wrote music for it. (Wilson has an uncanny resemblance to Tiger Woods).

The second half of the film touches a little on the software process, in particular, it talks about a discussion of a bug, and whose fault the bug is. It also talks about the criticism the whole project has had, although frankly, I think Project Aardvark is a brilliant idea for what it's trying to do, which is lure top-notch talent to Fog Creek. You get a sense of the success of this when one guy talks about the sense of abandonment he's feeling when he leaves this project, and that he's going to work on meaningless school projects.

There's also a "passage of time" segment as one guy shows his tomato plants up on the rooftops of the building they work in. You get to see the tomatoes about a month into the project, all the way to the end, when the guy hands out tomatoes to the employees at Fog Creek, and in particular, to a bemused Joel Spolsky. (If you watch the credits, you can see its fate too).

As another note, one thing you realize is there are hardly any women. In fact, only one woman figures at all into the film, and she's not a software developer. There's also barely a person of color. At best, it appears one guy might be middle eastern or Latino, and one guy of possible Russian descent. There aren't African Americans. There's one Asian American. Not even Indians, who tend to figure promiently in the software industry.

This is a problem you see when you deal with companies looking for top-notch talent, especially small companies. There just aren't many talented women or minorities in computers. It's not that these companies don't want to hire them. They'd love to find some woman or some African American or Latino who is a brilliant coder. It's just that they are extremely few and far between.

To give you an example, how many top Asian Americans are in football? Three, four? Asian Americans make up, say, 1-25 of American population, but probably make up less than half a percent of NFL players. African Americans, who are maybe 14% of the population, make up at least half the numbers. This may be cultural. This may be genetics. Who knows? But the point is that when you look for the "best", you can't get good demographics, and that's trying to be perfectly fair to everyone.

At the end, there's a bet to see how long it takes, when they go live, before they get their first sale. I suppose that makes it the climax of the documentary.

The documentary closes up, much like you'd expect, which is to show empty cubicles, computers with screen savers, i.e., a kind of closure to the process. When I get the chance, I'll probably watch it with the commentary, but you have to realize these kids aren't professional actors or directors. Even professionals often do a horribly job with commentary, because commentaries are often done impromptu, often live as the people are watching, rather than scripted. Needless to say, it produces a wildly uneven results with some commentary being brilliant, and others nigh-well unbearable to listen to (for example, the cast commentary to Terminator 3). I'm not expecting a whole lot.

In the end, I think this might have worked better as a series of half hour segments, done week by week, where we actually learn about the code, about the problems people have working together, about how the whole thing was planned, and how much was interns deciding stuff, and how much was full timers deciding stuff.

What I liked about Aardvark'd: the sense of New York, that Fog Creek looks like a cool place to work, Joel Spolsky, the music.

What I disliked: hard to tell who was who (the documentary struggles with this too, often showing one guy juggling, so you know who he is), little about the software development process, very little about the guys at Y combinator, what makes these kids so bright, and what you miss out when you don't get bright people.

The production was at times good (especially the end credits), and at other times, rather amateurish. You can see a struggle between the kind of film Wilson wanted to create (it's about the people) to the kind that Fog Creek may have wanted (it's about the people, but as coders). The question is who is this film pitched at. If you want to use it as promotional tool to attract brilliant folks, does it really work? I suppose it could, if you say brilliant coders like their limelight too, and see that the work environment isn't some stodgy office filled with incompetent nincompoops with busy work.

Anyway, I think it was worth the money I spent on it, although I was expecting a different documentary altogether.