Innovation in Software

Source: Wikipedia

Somaliland is an autonomous region that is probably very much like your pre-conceived notions. Its 3.5 million people have struggled through warfare (many suffer post-traumatic stress disorder), the economy is “in early stages of development” and it has suffered greatly to gain recognised independence.

Few people know of its existence, fewer still care about its future. But we should. The Horn of Africa has seen great misery and human suffering over recent decades, and whilst it is a country that has many faults including widespread corruption, it is at least a better attempt at democratic, peaceful governance than anywhere else nearby.

Alas, the entire country is now at risk, thanks to a technical “solution” to a problem that never really was.

The current edition of Private Eye has in its column “From Our Own Correspondent” a story from Hargeisa that should make all involved ashamed of themselves. Emphasis throughout is mine:

Somalialand is the only place in the Horn of Africa that is democratic, stable and tolerant. Yet because of misplaced fears of the mushrooming of micro-states, we remain unrecognised by the international community, 18 years after declaring our independence. As a result the world keeps us at arm’s length and has instead forced on us consultants so greedy and inept that the very peace we now enjoy is under threat.

Elections in an impoverished, nomadic society are never easy, but our record of closely contested polls compares pretty well with our neighbours [Somalia, Ethiopia, et al]. Our friends faraway nevertheless thought that what we really needed was a state of the art biometric finger printing and facial recognition system to compile a voter’s roll. But an operation of such complexity – not to mention the $10m funding – could not possibly be trusted to us natives.

[...]

Alas, this model [...] has somewhat underperformed. Presidential elections have been postponed four times now and are 18 months late, and now we have the prospect of civil war as our politicians cannot agree on a way forward.”

It’s a stable country that has a reasonable record on electoral fraud prevention. Who then thought that an advanced biometric system was what this country needed?

I’ve left out from the Private Eye piece the criticisms of how NGO Interpeace are (mis-?) handling this, how Britain and the US are washing their hands of it, and the details of who is blaming who, but the error was there at the start: they placed the country’s future in a technology system that wasn’t needed. No doubt it was profitable for somebody.

This is a country with a GDP per head of $226 – the vast majority of the population are living on less than $1 per day. $10 million could have helped address woeful statistics such as only 25% of Somaliland adults are literate, and just 17% of children go to school. The funding could have even helped the 72% of the population without access to clean water get some new wells.

But it gets worse.

The nomadic culture that dominates Somaliland (any two citizens can work out how they are related by sharing their names and clans), is culturally sensitive to finger-printing. That of course makes a biometric database a fatally flawed model. The problem they are attempting to address – that people from neighbouring countries could vote – has been “solved” with a system that introduces new problems, that means not even all of those entitled to vote want to necessarily register.

To cap it all off, the people who went about delivering the system didn’t just do a bad job at implementation, but a thoroughly awful one. According to some sources on the ground:

The current voter list is neither accurate nor can it form the basis of a fair and transparent election. The only alternative is to go ahead with the election without voter lists.
Interpeace stated in a controversial and a very contradictory press release dated July 25, 2009:

“The Voter Registration system was seriously abused during its implementation, with widespread corruption and systematic fraud, resulting in the failure to record the fingerprints of more than half of all registrants. In other cases, over 150 registrations were made with a single fingerprint at the same registration centre, or through photographs instead of in person.”

Say what? There are voters walking around with 150 polling cards, and others who got registered without ever actually turning up? That sounds like the kind of thing the project was meant to protect against. But the systems can spot all that and deal with duplicate registrations, right? Well, according to another source close to the action:

IT Professionals advised the hardware of the server should be upgraded and software to be reexamined to be fit to handle database of 4 million voters in Somaliland. The testing phase must include plan for next 20 years according to population growth rate.

They called for upgrade of both hardware and Software including the operation system, which should have latest security and performance tuning patches. The hardware upgrade should include Hard Disk, RAM and the Processor. They highlighted that majority of the data captured in the server are not in text [but in] binary format like picture and fingerprint, which needs massive storage area. RAM and Processor helps the server to boot and run quickly particularly during filtering the duplicates. [sic throughout]

Failures then, include:

1. The whole project has several fatally flawed assumptions under-pinning core choices
2. The money could have been put to more effective use elsewhere
3. The software was designed incorrectly and therefore its output is unreliable and can’t be trusted
4. The server infrastructure is under-equipped and under-managed so now they need to add more storage, more memory and increase processing power (and by the sounds of it, whack on a few service pack upgrades).

These are all failures we see in IT projects on a regular basis. If this were a new ERM or accounts platform we’d sit around the board room table, gravely shake our heads, talk about “lessons learned”, and put it all down as a bad job. We’d move on, avoiding the compulsion to try and “fix it” due to our own notions of “sunk cost”. This is in essence, like thousands of IT projects that have happened over the last few decades.

Except in this case, the consequences are more serious: the country now faces civil war.

As you go about your daily work rolling out technology to your clients and customers, you may not think that your potential failure to deliver will result in human fatality. However it will have consequences.

Iit’s important we think carefully as our industry takes a greater hold on the workings of civilisation and shapes ever more its potential, about what our lust for automation and control can do to lives if we fail to live up to expectations. Somaliland is an extreme – but sobering – example.

As an early supporter of NO2ID and long-term member of Liberty, Amnesty International, and others, I have heard enough stories about technology tampering with elections that even as an advocate of the power of software to improve society, I am perfectly happy that my native democracy requires nothing more than pieces of paper and some pencils to conduct an election. I sincerely hope that somebody decides it’s good enough for Somaliland too, before it’s too late.

I find it odd that there is so much hullabaloo over a tech site endorsing a political candidate.

Last week I gave a short class on software architecture and why thinking through a problem before turning it into software matters. I referred to two quotes (the first is on the back of my business cards):

“Computer Science is no more about computers, than astronomy is about telescopes” – Edsgar W. Dijkstra

“Software runs civilisation” – Bjarne Stroustrup

I don’t do this job because I like playing with syntax, compilers and interpreters – they’re just tools. I do this because it matters.

There is no engineering discipline, no craft, no art, no philosophy that can have as large an impact on day-to-day life as software development right now.

People will stand for hours in a street to hand over a significant percentage of their income for a phone that can help make their lives easier or more fun. A significant proportion of people working in the West will spend forty hours a week using software tools continuously. A kid in a dorm room can go from having virtually nothing to being on the front cover of news magazines in a few years just because he knows some PHP and has a business head on his shoulders.

This stuff matters.

However, we do not sit in a vacuum. We do not develop the cogs of society to shape it, we respond to its needs. We are part of the societal machine we serve, and we’re here ultimately to make money for ourselves, our investors, our employees just like anybody else. We need to look at the World around us and attempt to understand it. I spend at least 10-12 hours a week reading not just news, but books on philosophy, art and trends within global culture – how could I attempt to be a software entrepreneur and not do that?

So when Tim O’Reilly decides to endorse Barack Obama why should he be condemned for it? You don’t have to read his post, but as somebody who seeks to understand the trends within the software industry, why should he not attempt to have an opinion on the society it sits within and shapes?

Being able to predict trends within society – and responding to them – is critical to what those of us who aspire to be at the front of the industry do. If you’re not prepared to understand the fears, insecurities, hopes, dreams and aspirations of the World around you, what on earth do you think you’re doing in the software industry?

Maybe I’m biased: I like politics. If you like politics, you have to be aware of the US political scene to some extent. To not be would be like being into fast food and ignoring McDonald’s. As a politico, even a British/European politico, you find yourself idling away through American Political speeches (turn your speakers off/down) and when you hear a candidate speak with the same conviction you start to consider whether they are going to do something special.

Back in January 2007 when I was making predictions for the year ahead which I mostly got wrong I added one prediction for 2008/09 in the last paragraph:

“Barack Obama will become the 44th President of the United States”

This was nearly a year before he announced he intended to run, which when it happened came as a surprise to many. At the time I made that prediction he had just two years in the Senate under his belt. If things go as the polls suggest they will, this might prove to one of the best long-range predictions I’ve ever made. Did I capitalise on it? No. Will it affect me in my daily life? Maybe. Will I be disappointed if McCain wins? A little, but he’s a left-leaning Republican anyway so Europeans aren’t as scared of him as they are his VP candidate.

What is important though, is it means I’m getting better at understanding how politics work, and that must ultimately mean I can get better at understanding how software works.

Those of you think technology and software sits inside a bubble, or that endorsement of one candidate over another is somehow “wrong”, simply don’t get it. You’re not in the same industry as us: you’re destined to only ever be consumers of ideas, never formers of them.

I read some pretty weird stuff out there. You might think it odd given I’m all into innovation that I rarely read the likes of Techmeme or Techcrunch – the truth is, there’s little innovation in either.

I want to look at the guys who spend days lying on their backs staring at clouds and thinking to themselves “that cloud looks just like a better way to moderate pseudonymous online discussions”. I prefer artist coders to mathematician coders: they have more interesting things to say. I’ve said this so many times it’s starting to sound like a cliché: I’ve learned more about software from architecture, art and philosophy than I ever have from my University notes on formal methods.

Occasionally, as I scour the websites of the research labs of the World, I encounter something a little bit “special” in the “let’s look at something unique and apply it to the computer industry” niche. This is no exception.

HP Labs have recently published (only appeared in my RSS today, but it’s dated the 12th July) a paper on the election protocol the city of Venice used to elect its ‘Doge’, or Duke/Chief Magistrate, if you prefer. It appears to have been published in the IEEE proceedings for their 20th Computer Security Foundations Symposium which happens to have taken place in Venice, 6th-8th July this year. It could all just be a bit of an in-joke.

However, there are some really interesting ideas in here.

The election of the Doge was – in modern terms – quite absurd. It consisted of 10 rounds of voting, and at the beginning the entire electorate (the Grand Council of oligarchs who were all men over 30) was eligible. Each round was alternately a random lot draw, and then an election. There is an outline of the process given in the paper, and you can see how long it must have taken to elect anybody.

Where the authors (Miranda Mowbray and Dieter Gollman) take it is to show that there are some qualities it would be sensible to consider when designing computer-based election procedures. Specifically they argue that consensus in asynchronous systems, recovery after network partitioning or indeed any scenario where a ‘lead’ has to be taken by some node might be all processes that benefit from such a system. Whilst absurd to us now, it has the virtues of stability and resilience to ‘gaming’ or corruption.

There are some real gems in here as they draw parallels between modern day computer security and this ancient electoral system. Part of the protocol required for a random ballot to ensure that the drawing was fair, was that the person doing the draw would be selected as “the first boy seen after prayer’s at St. Marks” by a particular member of the Council. Mowbray and Gollman describe releasing a young boy into the square at just the right time as a “protocol attack”. They go onto describe the protocol’s arduous and drawn out procedure as “security theatre” – something that anybody who has looked at recent anti-terror legislation is all too familiar with.

They go onto propose a slightly simpler protocol that could be used by computer systems and I’m left with a mind whirring with possible applications. This is the kind of paper that makes me wish I’d got the main Vagueware application online as I can suddenly find myself thinking of applications in search, auctions, community building and more.

This is what I mean when I talk about innovation in software.

Real innovation in this space is not about taking something you’ve seen on the Web and doing it slightly differently like thousands of others. It’s about taking a piece of politics, art, theatre, humanity, and finding something in it – a tiny piece of inspiration – that gives you an idea of how to make software better.

It would appear, rather controversially, that whilst just a year ago the Government said there was no interest in introducing e-Voting to the UK at all, the Department for Constitutional Affairs has invited councils to submit proposals anyway.

Proposals from councils are due in for the 17th November, which would suggest those planning to go ahead have probably already had preliminary discussions with software suppliers. The benefits the DCA say they are keen to see promoted in the pilots for e-Voting include:

• the system may assist users to avoid mistakes and prevent them inadvertently spoiling their ballot papers; This could be particularly useful where elections are combined and with complex ballots.

As the pilot is for a local election, I’m not entirely convinced ‘complex ballots’ will be around. In addition, most HCI experiments with e-Voting systems to date have shown that many computer interfaces are actually more confusing than a list of party symbols, names, party names, and a box in which to draw an ‘X’. There is even some cases where Diebold appear to have been implicated in making an interface deliberately complicated.

Imagine the following:

“Do you wish to vote for X or choose somebody else?”, options “Yes”/”No”/”Cancel”. It’s subtle, but a lot of people who fear having made a mistake would at this point get confused.

• the system may be used to link to the elector to candidate information to ensure they have the opportunity to update themselves on candidates and policies at the time of voting;

This is extremely dangerous. The potential for a HCI issue here is massive, and we have laws/rules preventing campaigning within certain distance of a polling station for a reason. I would be very interested in understanding where the DCA is heading with this.

• the systems may be configured to provide options for different languages – something that is obviously beneficial in an increasingly multi-cultural society where English may not be the first language for many people entitled to vote in elections;

This, I conceed is a major benefit of e-Voting systems. However, in areas where there is an expected high turnout of non-English speaking voters, I wonder why we aren’t providing translated ballot papers already. I also wonder how this fits in with the current Government policy of encouraging all citizens (and therefore all people entitled to vote) to be able to speak English. That relates in turn to how a voter who can not speak English well enough to vote could understand the campaign and debate. Shouldn’t this be a broader issue discussed within society before we try fixing it with magic flashing boxes?

• the systems may be configured to accommodate special needs for people with disabilities – this may enable many people who have previously been unable to vote in private, in person at a polling station to do so;

Again, massive potential for a HCI error here. Getting disabled access to IT equipment right is notoriously difficult. I am also not convinced that the considerable expense of rolling out an e-Voting system sophisticated enough to help disabled voters could be justified by providing a benefit to just a small percentage of the population. What’s more, I’m pretty sure that with some thinking it would be possible to provide that percentage of the population a means to a private vote without that level of expense. Would it really be that difficult to provide ballot papers with braille?

• e-voting allows quicker tabulation of votes cast and links with e-counting to produce a result more quickly. This could be of particular benefit in mitigating the increased time that is likely to be spent on processing postal votes as a consequence of signature checking and which may cause counts to be delayed to the day following polling day.

This is the main reason touted for e-Voting across the globe. Feels compelling, but you have to ask yourself which is more important: getting the vote right, or getting it quick? The main criticism of e-Voting is the ease with which it is possible to commit widescale voting fraud. Diebold machines are reported as having an easy way to completely change the number of votes cast after the vote, so what does it matter if the count is quick if it’s thought that it could easily be compeltely made up?

• e-voting may have a positive environmental impact by reducing the need for paper. There are however other factors that need to be considered to identify whether there is an overall benefit in this regard.

Using recycled paper and then recycling cast ballots after the statutory time they need to be retained would be more environmentally friendly. e-Voting systems are composed of horrible toxic chemicals – the contents of a hard-drive would make you scream – that are difficult to recycle, and they of course require electricity to operate on. Paper doesn’t.

• the electronic storage and management of data may improve the contingency arrangements available to electoral staff.

Direct electronic storage and management of the data are actually completely open to abuse. Did you know that if you store data on a CD-R, it is reckoned to start corrupting at a bit level within just four years? Paper records last 100 years or more – there is no electronic medium that can get close to that currently.

The report also suggests pilots for ‘remote e-Voting’ via the Internet or phone. It shouldn’t take a genius to realise that this is open to massive, wide-scale fraud and attack, far greater even than with postal voting. If any council, anywhere, goes down that route, I’m afraid we have reached a point where we no longer effectively live in a democracy, but rather one where the person with the most resources available to cheat will be able to do so for less money than ever before. Voting is not like using a credit card to buy a book, or to vote on Big Brother – the mechanisms are too different to make them comparable.

Anybody who thinks that Internet voting has a future, should consider why the developers behind GNU.FREE decided to stop production in 2002:

’From my experience of designing and developing GNU.FREE over the past three years it has become clear that creating an Internet Voting system sufficiently secure, reliable and anonymous is extremely difficult, if not impossible. As Bruce Schneier points out “a secure Internet voting system is theoretically possible, but it would be the first secure networked application ever created in the history of computers.“‘

When developers, backed up by Bruce Schneier point out that you’re against the wall on this one, you should probably take notice.

It might seem strange that as somebody who touts innovation and technological development, I am posting an article ripping e-Voting to pieces. I understand the benefits the DCA is looking for, but over the last decade I have seen serious problems with this technology and feel that it is therefore my duty as somebody who understands those problems at a software level to point them out to the electorate and the officers within councils responsible for overseeing these votes.

It’s important if we go down this road that I and others who write systems like this explain to all concerned what those problems are, what the risks are, how to counter them, how to reduce them, how to avoid them.

I’ve already sent an e-mail to my local Electoral Services Unit to ask them if they have made a decision to run a pilot, if so with which suppliers, and if they would like me to come in freely and explain what problems they might face. I suspect the answer to the last offer will be a firm ‘no’, but I’ll keep you posted.