Some random thoughts on programming

Since this post actually got read by people I made a follow up to clarify some of what I was saying. Thanks internet for helping me realize how bad I am at making my point.

As I near the end of my third decade programming, I keep finding myself coming back to pondering the same kind of questions.

• Why are some people good at this and some are not?
• Can the latter be trained to be the former?
• What is the role of language, development environment, methodology, etc. in improving productivity?

I’ve always been partial to the “hacker”. The person who can just flat out code. For some people this seems to come easy. I have been lucky enough to work with a few people like this and it has been awesome. As you talk with them about a problem they are already breaking it down and have an idea of how the final system will work. I don’t mean that other aspects of professionalism, experience, etc. aren’t important its just that I believe in the end there is a raw kind of talent that you can’t teach. My wife can sing, I can’t, and not for lack of practice.  She practices, but she can also belt out a perfect tune without hardly trying.

What are the core talents that makes someone a good hacker? I think there are two:

1. The ability to break down an idea/system into an abstract set of “concepts and actions”
2. The ability to understand how these “concepts and actions” execute in a particular environment.

Note that I didn’t say anything about coding. Coding is actually the least interesting part. If you can do the others well, coding is just another kind of typing. Coding is the thing your fingers do while your mind is flipping between 1) and 2). Learning a new language is less about learning the syntax than it is about learning the execution model.

Writing good, correct code involves not just conceiving of how something should be 1) but also understanding all that can happen 2).

I honestly don’t know if these things can be taught.

In 30 years I have only seen a few examples where a mediocre programmer became good or a good one became great and zero examples of a mediocre programmer becoming great.

What does this mean for people who want to build large software systems? The easy answer is to only hire great people. Which is fine when you can do it but not everyone can be or hire the best so what is left for the majority?

What should be done to make things easier for people aren’t in the top 5%?

There are lots of nice things that have been invented like source code control, higher level languages, garbage collection, etc. which reduce or abstract away some of the problems. And while sometimes these abstractions are leaky, they are really one of the few weapons we have against the beast called Complexity. Complexity is the key problem we face, especially if we are not wizards because:

A persons talent with regard to 1)  and 2) above determine how much complexity they can deal with.

When I was in gradual school at UNC-CH Fred Brooks (the mythical man month guy) used to talk about Complexity as being a major thing which makes programming hard. Its not something easily cast aside:
Quoting Dr Brooks:

The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence.

I wish I had appreciated everything Fred said when I was a cocky grad student as much as I appreciate it today… sigh.

Some things are hard because they are hard and not much can be done about them. Still our goal needs to be to not try to make it worse than it is.

To this end, some people say:

Do the simplest thing that will work.

I agree but prefer this formulation:

Strive for the simplest thing that can’t possibly fail.

If you think about what can go wrong you are more likely to find the problems than if you think about what will go right.

I wish I could say we were making progress in reducing complexity but we seem to be adding it into our programming environments rather than removing it. One of the things I have thought about recently is the complexity of these abstractions themselves. Programming languages of today are so much more abstracted and complex than what I learned on (assembler, basic, pascal, C).

For example the world (or at least the web) is currently engaged in writing or rewriting everything in Javascript. Not because JS is so awesome, but because 15 years ago it got stuck into browsers, then we all got stuck with browsers and thus we are all stuck with JS. Sure there’s lots of server stuff done in various other languages but really how long can that last? How long can one justify having to write 2 parts of an application in 2 different languages?

The future looks like node.js.

I am not a JS hater. Its just that I used to think Javascript was a simple language. As someone who wrote 3D Game engine code in C++, my thought was “if these webkids can do JS then it can’t be that complicated”. After building a webapp using Javascript/Dojo/svg to create a realtime browser for genetic information, I realize how wrong I was.

Sure its all nice and simple when you are doing this:

$(“#imhungry”).click(function() { alert(“food”);});

However a lot of complexity lurks just under hood. And its the way it all interacts that concerns me. Crockford called it  Lisp in C’s clothing. This should excite a few people and scare the crap out of the rest.

JS has lots of opportunity for expressing yourself in interesting ways. Expressiveness is good, but too much of it can create more problems than it solves. Back when C++ was a bunch of source code you downloaded from at&t and you had to compile the complier yourself before you could use it, we grad students stayed up late eating bad fried chicken and debating the merits of O-O programming. It was at this point I began to wonder if the productivity gains of O-O were more the offset by the time spent debating about what good O-O practice is. This tape has played in my head more than a few times in my career.

Its not all bad of course: On balance, garbage collection is a win and I see lots of benefits to using closures in JS, but the underlying complexities of things escape most people and their interactions are understood by even fewer. If you have to slog through this (and you should) to understand what happens when you call a function, you are cutting out alot of people from fully understanding what is going on when thier code executes.

I hope that some of this will be address by standards of practice. Things like Google’s guidelines seem to be a good start.  As with C++, I think that the best hope for improving productivity will actually be to agree on limits on how we use these tools rather than seeing how far we can push the envelope.

Well except for wizards and we know who we are 🙂