Innovation in Software

Michael Kavis has posted up a short note on The Lost Art of Software Engineering and asks whether we have we lost the skills that once dominated the industry. Are we just looking after the business side of things, and is that wrong?

As somebody whose degree was Software Engineering – as opposed to ‘Computer Science’ that dominates the UK Universities – I feel that I can give a little insight into this.

First of all, I believe he’s talking about the wrong thing. Software Engineering is literally applying formal methods to the process of development in order to make sure that it always behaves in a predictable manner. The software can fail, but it should fail in a way you expect and have accounted for in terms of providing failover or re-instantiation (i.e. you can reboot without killing anybody). Typically, it is used for designing fly-by-wire systems on aircraft, management of nuclear power stations, systems that have a direct effect on health like life support or MRI scanners, or for embedded systems that might be difficult to maintain such as satellites or marine equipment.

Software engineering is not about transactions per second or response times unless those are business requirements. Usually software engineering is purely about applying formal methods to ensure the most critical business requirement of all is met: nobody’s death is caused by the system.

This leads me to where I think Michael has taken a wrong turn: the quantifiable measures he talks about in his post are not showing up in specification or tender documents because the business requirements don’t require them to. There might be a point that business requirements aren’t discussing system requirements like response time enough, but we are now at a point where we are dealing more with humans and their requirements than we are computers and their requirements.

This isn’t something we should be ashamed of. We now have the memory, CPU and tools to be able to build software that fits around our environment and we should embrace that. It’s actually quite hard to build a web application that doesn’t answer within a second, and if it does become an issue we then use the most useful tool in our box: re-factoring. Never optomise too early, scale when you need to. The business requirements in most cases is to get something shipping, and fast.

Over the last few years there has been an upswing in the interest around Agile and specifically design principles as they apply to software: as programmers we are beginning to understand our work at a philosophical level as much as we are a technical level. This is going to lead to developments that we couldn’t dream of a few years ago, and it’s because we’re freeing our mental capacity from having to worry about malloc() and free() every few minutes.

Where the skills we have as engineers come into play is making sure that the business requirements are understood and expressed in the simplest possible system. Instead of thinking about responses per second, we are free to concentrate on model validation. Instead of mathematically proving an algorithm will always work, we are building automated test frameworks. We actually spend more time engineering and less time hacking than we did just three or four years ago.

Software engineering isn’t a lost art, and neither is systems engineering and design. They might have evolved, but they’re far from lost: they’re just getting started, but now they’re much more hands-on.