Innovation in Software

I read Joel’s article on Undergraduate CS classes with interest. I’m one of those people who genuinely think the next generation of software is going to suck, because the current generation of teaching is absolutely awful.

Right now people are seeing software development as something akin to mechanical engineering that you should study not one day before the age of 18 when you arrive on campus. Even then, we teach software development the way we would teach applied physics in a civil engineering class: teachers who haven’t worked on a commercial project in their entire lives are convinced that the books they contain – the books full of answers – are accurate.

They have no idea that software development – like any other sizable discipline that is ultimately rooted in aesthetics and philosophical understanding – is about questions, not about answers.

The quality of graduates right now makes me angry. My blood pressure goes through the roof whenever I see prospectuses offering to take somebody without a shred of programming experience into a qualified developer able to manage a team of other developers in just four years.

If we got it right, we could start teaching other subjects – from art, music, philosophy, through to physics and maths – through the discipline of software development as a tool from as early as 11 years of age. We don’t though, nor will we ever. It doesn’t matter that modern civilisation runs on software, our educators don’t understand that and therefore assume the children they teach won’t need to either.

I have ideas on how to fix this, but they’re flaky and need substance. The only concrete detail is that real practitioners out in the industry need to take the lead on this one.

On the back of my business cards I have 10 quotes which on discovering them the first time, I found to be something that resonated with me, and that I hope might resonate with potential clients, business partners and friends.

The first of those is a famous quote by E.W. Dijkstra that for me sums up the reason I got into the industry in the first place:

“Computer Science is no more about computers than astronomy is about telescopes”

I also recall Ted Nelson’s talk about Transliterature at OpenTech 2005, where he also summed up why computers fascinated me as an 11-year old learning to program the first time:

“I studied Computer Science to help change the World, not to automate trivial crap”

There is something bigger here in our industry we refuse to acknowledge. There is something deeper beneath the surface that all the talk of social networks, long tails and user-generated content doesn’t get anywhere near.

This ember of a notion has been inside me for a while now, and it’s starting to turn into a small fire. I don’t know where it’s going, but what I do know is that I’m now getting more and more passionate for “big picture” stuff. The kind of things that need investment and great people.

I’m rather pleased then, with all this “big picture stuff” going on in my head, that this year’s Turing Lecture is being held again at Manchester University and that it has just been announced as being given by James Martin, producer of the film Target Earth – note, not the 1950s B-Movie, alas! However, it’s big in its approach, and I’m looking forward to watching it just before Dr Martin gives his talk.

I still haven’t decided what 2008 is going to be about for me professionally, but I do know it’s going to be less about me and finding ways to reconnect to that Dijkstra quote in my work. The Turing lecture will be a timely reminder of some of the issues facing us – and maybe sometime this year those of us in Manchester can start thinking about how to work out some of the solutions. Maybe.

I’ve just decided “Maybe” is my new favourite word.

Happy New Year.

”It is true some rather interesting projects get abandoned by their developers or shall I say their artists. Most of them are sculptures made out of C (or any other language). And this is the biggest handicap of those programs. They aren’t engineered they get crafted.” – Reiner on www.gnomedesktop.org

When I was 11, I wandered into the school library and on the shelves found a book called “Learn to program BASIC on the BBC Micro”. It was about 60 pages long, and most of each page was covered with pictures of robots telling me what ‘PRINT’ did. Coincidentally the time of year was Lent, and it being a Catholic school I had the option of foregoing lunch and instead donating 40 pence to CAFOD to use the computer room for ‘personal projects’. This was how I began my study. It quickly became an obsession.

By the time I got to 14, I had discovered a few things:

• I couldn’t afford a real computer, but it didn’t matter. This was 1992 and yet whilst my richer friends were playing with Amiga 500 and Atart ST machines, I was stuck with an Amstrad CPC 6128 my father had kindly managed to donate. This was where I learnt the true value of engineering within constraints.
• I didn’t want to study Computer Science. This, I still believe was the right decision. However, my failing was thinking that I wanted to study Software Engineering. I didn’t want to be an academic trying to understand the philosophy of bubble sorting – I wanted to change the World and write software that would help run The Future, and I figured Software Engineering would give that to me.
• My school teachers had realised I was the best person to call when a computer broke somewhere in the building, and as a result, I knew I would probably be able to make a living out of my obsession if I needed to. Knowing that you can make a living with the skills you have when you’re 14 is… empowering.

I set my course. I worked just about hard enough (but no harder) to get to UMIST – now merged into Manchester University a campus I still live on the edge of – and studied Software Engineering.

Actually, that’s a lie.

I was on the course, sure, but I chose to get drunk most of the time instead. See, the problem was, fine, I could learn formal specification and learn how to build a compiler, and that would all be swell, but I now realised it wasn’t going to change anything. I was not going to be able to express myself the way I wanted to if I was confined by determining the correctness of my code mathematically. I thought Software Engineering was more concrete than Computer Science, but in actual fact it was just the same abstract notions but sat in front of a screen and expressing them via C instead of in the library expressing them via pseudo-code.

What I then realised is something I should have had the guidance of when I was 14: I didn’t really want to be an engineer. Engineers do what they’re told, follow a plan, make sure the foundations are set correctly and regulations are followed.

I wanted to be an architect, a craftsman, an artisan. The depression of the realisation that I had taken the wrong course actually drove me to the point of barely turning up for class at all. I ended up taking on a variety of full-time jobs whilst attending exams so I at least came out with a piece of paper. I also spent a lot of time getting drunk and playing pool.

This conflict still continues inside me – sometimes I feel I want to be the engineer, building automated tests, writing out my code in formal proofs, knowing that my code is correct. Other times I want to be the scupltor, using the IDE like a blank slab of marble or a lump of clay awaiting moulding. Some of my best programming has come from just writing a line that kind of looks like it might be the core of what I wanted to do and then wrapping it in other code that made it work. That’s hardly the way you would want a fly-by-wire or power-station safety system to be written, but it is immensely satisfying. It’s fun. It’s art.

So now I look at these formal methodologies, whether they be more traditional lifecycle-led with their heavy requirements capture stage at the front, or if they’re agile, and I think “what a bore”. In some ways I’m drawn more to agile methodologies not because they are more productive, but because they are more fun. They are there to provide confidence to the client, sure, but they allow the artisan programmer out, free to roam.

It feels wrong though – when I go out roaming, I feel as though I am not a professional programmer. All of a sudden I feel I am an artist playing amateur programmer, and I live in fear of being discovered and found out. I constrain myself with project management tools and development methodologies, not because I think they do me any good, but because they stop me feeling guilty.

You know how I spend most of my day? I write a lot of Ruby, specifically Ruby on Rails. I reckon I must spend about 20%-25% of my time looking things up in the API docs for either Ruby or Rails. Do I know how to code in them? Sure I do. I just don’t bother memorising the API. I just find the bit I need, the syntax and method parameters that need to be called right there and then. With time some methods become second-nature, but I don’t actively memorise this stuff. I’m too busy coming up with my own ideas to remember other people’s. How un-engineer-like is that? How skittish and flaky is it? I don’t think I even deserve to be called a programmer if I do things like that.

But I bet I have more fun than most engineers. Because as the code comes out, at the end, it feels like a piece of art is being formed. Sure it might be a reporting module that spits out graphs and pie charts, but I know the inner beauty of that little bit of optomisation that made it query the database 12 times rather than 800 times. I know how I flitted from the model, to the view, to the controller, hoisting each little function up like a piece of scaffolding. I know that before I started there was nothing, and now there is code, there is function, there is value. The World is very slightly different for the users.

If I could go back and give advice to my 14-year old self on how to become a professional software engineer, you know what I’d tell him? Go and study Art or Philosophy. It will do more for your ability to create than any Computer Science course ever will – and you can learn all the CS stuff from Knuth anyway, if you need to. I’d tell him that the true beauty of this profession is that it has the advantages of being a writer, a musician or artist except it’s far more profitable. I’d point out that understanding how code works isn’t as important as understanding how you work.

I’d also probably tell him which girls secretly fancied him and to stay off the beer when he got to Uni, but that would spoil the journey of excrutiating self-discovery. :-)

So I’m keen to learn: am I alone in this industry? Should I be stripped of the title ‘Programmer’ or even more laughably ‘Software Engineer’ and just be known as ‘that guy who does that stuff, with you know, the computers and stuff’? Is the real motivation behind the uptake of Agile not the fact it makes people more productive, but it makes people feel more natural, more like artists? What would you tell the younger you about how to harness ‘the art’ or the ‘the science’ of programming?