Author, Author

I want to write a book. I'd like to tell you that this book will reinvent some genre, or even what it means to be a fiction book, or that I'll have some keen insight a la Jared Diamond. I'd like to, only I can't, because the book I'm thinking about writing is far more mundane. I want to write a book about programming.

Thousands of books have been written about programming, and I'm sure thousands more will be written. The real problem with programming books is whether it adequately prepares you to program in a software development environment. Most books written are aimed at beginners. These books are the easiest to write precisely because there are so many out there to begin with.

There are economic reasons to write books for beginners. There are simply more beginners than advanced programmers. For every beginner that succeeds at learning programming, perhaps two fail. Of course, I'm completely making these numbers up as I have no stats to refer to. However, they reflect the intuitive belief that few people are able to master programming well enough to make a living at it.

Beginning programming books are intriguing for another reason. They're the first time you get to "meet" a beginner, to introduce them to the idea of programming. You can present programming to them anyway you want. Maybe you have the magic way that gives insight to someone, and they see the joy in learning how to think like a programmer.

There is reason to be discouraged. If programming were so easy, wouldn't more people do it? The answer is probably no because programming can be frustrating. Apparently, Donald Knuth, a "famous" computer scientist (he is famous within computer science circles, just not famous the way Ben Affleck is famous) claims that only a small percentage of people think algorithmically, which means, thinks in a way that they can solve a problem by writing a program. That percentage is something like 2% of the population.

I've been programming for a few months now in what seems to be a typical software company. Perhaps they assume a little more about what I should figured out, instead of providing me with the kind of guidance I want, but the point is not that. I'd love for my job to be all about developing algorithms. This assumes that the difficulty is not the programming language itself, but in the creation of the algorithms.

Perhaps an analogy is in order. An architect might design a building without too much regard for the materials the building is made from (although material has a profound influence on the way a building can be built, and these days, with computer programs, you can be even more creative with the material you do have---see Frank Gehry). You think about its design, and just design it.

In programming, you'd like to think of a solution that can then be translated easily to a programming language of your choice, be it Pascal, Lisp. Java, or even, Prolog. But algorithms tend to be about abstract things like graphs and sorting. What if you have to worry about sounds and images and wonder why you have to do such arcane things to get it to work. Do you have the correct drivers? The correct codecs?

This is, in fact, the thing that's most frustrating about programming. We're now able to do more and more with programs. People can write their own programs to simulate a Tivo, or to make some kind of interactive map, or even edit images or music, or what have you. Yet, to get these programs to work, you must know some arcane details that are often not well-documented.

It becomes less about algorithms, where you understand the language, and only need to solve the problem, and more about learning the language. Imagine you wanted to write a book, and every few chapters, you were told to use a new language. First, it's French, then Spanish, then Chinese, then Thai, then Hindi. Each time, you wonder how verbs can be conjugated in yet another way, and wonder why there are so many exceptions to the syntax, and why some languages insist on incomplete sentences while others insist on it. Why do languages have levels of formality, where you say one phrase to a peasant, and one to the emperor? You just want to tell the story, and you have it in your mind, and yet, you fight the language for no good reason.

Is that really algorithmic reasoning? Is it?

So, I want to write a book for beginning programmers, and yet, I'd like to give a sense of how far they need to travel before they get good. Understanding how to think algorithmically is only the first step. Dealing with the plethora of technology around you, and learning how to make sense of it, and use it intelligently is just as challenging, and has little, if anything, to do with thinking algorithmically.

The real problem with writing a beginning programming book is that many of the readers aren't even good at thinking algorithmically, even with lots of training, and yet, that's who you're teaching to. And many don't even like technology because it's filled with arbitrary and rigid rules. Arbitrary in that the rules don't make sense, they just are, and rigid in that if you fail to follow them, you fail to get a working program.

What do authors do? They simplify. And they simplify. No sane programmer (ok, few sane programmers) would use anything besides an IDE or at least a very good text editor to write their programs. Yet most IDEs are notoriously complex, offering hundreds if not thousands of options to set. Options for features you never knew existed, and never knew that you'd need them. The IDEs aren't even self-explanatory. You need a book, and then a tutor to explain which of the thousand features you should learn first, since books only offer a laundry list of items that exist, rather than tell you the important ones you need to know to get started.

Because of this complexity, there are some authors who say "Pish posh! That is so hard to learn, that students shouldn't learn it at all!". The expectation is that beginners, once they become suitably advanced, will choose to learn the more complex IDEs that they've been shielded from. Except most people don't like to become more stupid before they become smarter. This means that it's tough to wean experts of a simple text editor, and move them to a more complex IDE.

In fact, that's really the big problem with technology. Someone's always coming up with something new, that works so completely different from what you expect, that people, even very smart people, refuse to learn it. Because once the do learn it, someone will come up with a "better" idea, and the whole cycle begins again.

I talked to two guys who are writing a book and asked if they were going to talk about UML, or to talk about debuggers, or a version control system. Many of these are used every day in real software organizations. They said no, that you can't teach everything in a book, especially a book for beginners.

So why don't they write that book? Because it's technology. In order to make any sense of it, they'd have to pick a specific technology, like CVS, and then they'd have to hope that companies still use it years from now, which is perhaps less likely than a company using Java years from now. Furthermore, it's boring technology. There's not much in the way of algorithms or cool ideas. Yet, version control is not that easy to learn. It has the double whammy of not being interesting and fun to write about, and also not being easy to figure out (just explain branches, branch merges, diffs, and tools to look at a CVS repository, if you think it's easy).

There are plenty areas of programming just like this. Continuous builds. Ant. Make files. Maven. Debuggers. Profilers. These are tools needed by real programmers to write real programs, and yet the average programming book author doesn't want to touch these topics with a ten foot pole (why don't we use metric?).

Tools get obsolete. They're tedious. They're hard to learn. The new version won't work like the old version. Programming, as ephemeral as it may be, feels a lot more solid than the tools needed for programming. There's a sense that if you understand algorithms, you're going to be much more successful learning a new language.

And still, I want to write a book about programming, even though I should really write a book about tools, and how people use tools. I should see why people write buggy programs, and use that knowledge to create languages that are easier to learn, and yet this idea seems foreign to everyone. It's hard to create new languages. It's hard to make people want to use new languages. It's hard for new languages to do enough. Can you use your new language to play music, display video, play games, hook up to the Internet? Or can it only do calculations, and solve Dijkstra's algorithm? Is it enough to do "real" things?

I haven't even gotten to a major problem with teaching programming. We still use books! I love books, and yet I hate them. Teachers use books so they don't have to lecture for 20 hours a week, and so students can go back to the book years after the course ends (though these books are obsolete years after the course ends). Good books allow bad teachers to get away with it. Yet, books require a kind of patience most students don't have, and often teach in a way that students don't learn.

Most books are good as references. You want to learn about loops? Here's a chapter on loops. Want to learn about arrays? Here's a chapter on arrays. You could have a book that teaches you to solve problems, and then the book might have to cover a little about arrays, loops, functions, all at once. Maybe that would make sense pedagogically, but when you came back a year later, it would be hard to find anything in the book.

Thus, there is this dichotomy between wanting to find a pedagoically sound way to teach students (and that's assuming you can even find a one-way-fits-all approach) and yet make the book useful once a student is done learning the material. Ideally, you'd have two books: one to learn from, and one to reference.

And that's if we use books! Why aren't there more DVDs that teach programming? Why isn't it more interactive? Why books? And yet, why not? You might have a TV show on a DVD where an expert teaches you to program. That has some advantages. In particular, with students having short attention spans, a video format may make it easier to learn. On the other hand, when you need tables and charts, and tedious detail, text is more important.

I find it interesting how teachers feel pressure to make themselves understood to students, and that usually means they must remove technology barriers, because many students fear technology, and that fear makes them avoid learning about technology, which means they can't learn to program. Minimizing this frustration may help now, but what about in the long run?

I remember, once, sitting in a class. The professor was trying to present algorithmic ideas as simply as he could, trying to help the students along, and yet he completely rushed the technological parts, rushing over CVS, as if that part were easy. Well, open a book, Mr. Professor Man. Find me the chapter that explains sorting. And then find me the chapter that explains CVS. What? There is no chapter on CVS? I'll be! And you only spend two rushed (hurry, hurry, hurry!) minutes on it, while languishing slowly on how sorting works? Tsk. Tsk.

Look, I have no real answers on how best to teach programming, but I think I'm raising valid points about what's missing in programming. At times, I wonder how programmers have managed to survive this long. I yearn for the day when programming is about algorithms, but I think that day is long past. If we get back to that, it's because some area of computer science has become mathematics again, where the pragmatics of day-to-day use is not important, where tedious details are stripped away, where we can, once again, worry about what matters.