Innovation in Software

Adrian Holovaty announced a while back templatemaker which sounds more complicated than it is.

Take a pile of web pages all built using the same template – say, restaurant listings, or a timetable – and throw it at the code. It produces a copy of the ‘template’ with the ‘data holes’ marked up. You then look at a new page, and it gives you the raw data. No more screen scraping, it just learns what it needs to find the data that changes each time. Nice.

A really useful aspect is that whilst he’s implemented it with Python bindings, the core is written in C and uses the Python C bindings. That means with a bit of hackery, porting it to Ruby should be quite simple. A project for the weekend, perhaps?

I’m shouting this up for two reasons: it’s a piece of code I’m sure all of us have thought about implementing, but never got around to producing because it just ‘felt’ too difficult, so it’s definitely very innovative from that perspective; secondly, it’s using the C bindings in an intelligent way to get performance where it counts – something I’m going to need to do more of over the next few months in other languages.

I suspect the future isn’t going to be about using the right language for a project, but the right language for the right part of the job. You could implement something in Erlang, which can interface with Ruby, which in turn can interface with C. There are points of complexity there that a competent software engineer would shudder over, I’m starting to feel that being competent in one language or framework is definitely not going to be enough for the next generation of software: we need the flexibility and power of scripting languages, and the raw power of compiled C/C++.