One of the advantages of being almost 30 is that people less than a decade younger than you tend to think of you as being “wise”. Some of the staff in my local bar will ask me about everything from US politics, the Renaissance, Alan Turing, 1980s TV commercials and arcane facts about the early forms of Parliament. Cultured bunch, the staff in my local.

Last night however, it was my turn to learn. One of them had asked me last week about the phrase “Jumping the Shark” and where it had come from. Last night she told me the very next day after I’d explained it to her, she watched an episode of Scooby Doo (OK, maybe they’re not that cultured) where Scooby jumps a shark and that it had made more sense to her knowing what it was a reference to - it is one of the classic insider jokes within TV comedy. I then had to re-explain it all to the other people assembled. The conversation that followed was… interesting:

It went on in a similar vein for a few more minutes. More examples of the futility of the network, the silliness of the apps. Admittedly, none of them had left Facebook yet, but that might be that it’s rather hard to leave, as GeekUp and Co-working day regular Alan Burlison found out

These are people the social networks need. In their early 20s. University students. Bright, intelligent, aspirational. I have no doubt that within a decade most of them will be in the upper 25% of earners in the UK. Malcolm Gladwell would call them “sneezers” or something - they spread their likes and dislikes around their friends quickly. They set trends.

And in the last couple of months they have come to hate Facebook and MySpace.

Specifically, they hate that these networks have been opened up to people engaging in what is effectively a developed and sophisticated form of spam. They hate that they are being hassled via the social graph into doing “fun” things that are actually about as fun as receiving a hoax virus e-mail. They understand that their time and attention is important and its being wasted by sites that don’t respect that.

I have ideas for applications that will actually add value to the social graph and be of use to people in this group, but by the time I get to roll them out it could be too late - the people that make the platform interesting to me as a recruitment base for customers may have moved onto something else.

Facebook are adding features to improve the user experience as they learn how developers are gaming the system. They might win the battle in time, but ultimately they might have to give more control to users to block invites from apps that are not even remotely in their realm of interest.

This isn’t over yet. 2008 could easily be the year the social networks died.

This afternoon, I’ve been playing around with Facebook’s ad platform. Partly for Vagueware, partly for other businesses, I’ve been looking at what Facebook says about its user base to advertisers.

The level of targeting is just outright astonishing. It allows for ads not only to be targeted on demographics such as age range and city, but even on interests and relationship status.

For example, I now know there are approximately (all figures given are approximate to the nearest 20 or so), 120 people in Manchester interested in Programming.

Out of the 2,017,440 UK citizens who describe themselves on Facebook as ‘single’, 998,900 are male, 904,960 are female. The numbers don’t add up because some people don’t define a gender which makes the point that if you don’t fill info in, you can’t be targeted via that info.

There are 1,180 females in the UK who declare an interest in ‘Computers’. The figure for males is around 8,540.

580 UK men say they’re really into shoes, with 14,300 British women aspiring to be Imelda Marcos.

There are around 5,680 people working for BT in the UK on Facebook. In the US, there are around 40 people working for O’Reilly Media who mention it in their profile. I could target either with an advert - handy if you have a product or idea you want to pitch.

This is just the tip of the iceberg. Advertisers don’t need to know who your friends are (the social graph), to target you this tightly. If a member of GeekUp wanted to put up a singles ad for all single women between the ages of 24 and 32 who are into computers resident in Manchester (approx. 100 of them), they now theoretically could. Lucky ladies.

The question is, is this really a bad thing? Doesn’t it mean we’re not all going to see advertising that really has no relevance to us? Or does this kind of marketing mean that we are the perfect willing victims for advertisers to go deep into our psyche? I knew this day was coming, but I thought it was still some way off.

Vagueware is not my only gig. My other Directorship is very low-key, doesn’t take much time, and is where I and my business partner experiment with various marketing revenue models. It’s primarily been a learning experience, and the frustration we’ve had over the last two years in getting various complex projects rolled out has meant we’ve been looking at partnering with technology companies and focusing on the marketing and customer communications side.

Our latest venture is quite a departure for us. Excuse me whilst I shill for a couple of paragraphs:

Whilst cash back websites are not brand new, we’re hoping that with a really solid technology platform underneath us, we’re going to be able to do something special in the way of helping people make shopping a little more fun - and save cash too. I do however, need some eyes and ears because I’ve had no control over technology roll-out, so I’m interested to hear of problems people might have.

As you can see, the concept is really simple. You sign up, we give you a fiver. You shop online, we give you a percentage of what you spend. You refer friends, we give them a fiver, and we give you a fiver as well once they qualify for payout. We’ll point you in the direction of freebies that pay you money as well. We’ll be launching a blog to highlight particularly good offers. Occasionally e-mails with super secret codes will land in your inbox and you will consider yourself a wise old bean for signing up with us. It makes things cheaper if you’re doing a lot of Christmas shopping online, although for various reasons we’re late to the party for that one, so our strategy is a little more long-term.

End of shill

I mention it here, because I’m interested in problems an educated audience (that’s you, dear reader), might see. We know for example that the back end systems are rock-solid and everything is nice and secure, but are there ‘quirks’ we’ve yet to spot that only a geek can spot? Maybe you just think the business model is odd, or we haven’t explained it very well. Either way, I wanted people whose opinions I respect to take a look before the big marketing push over the next 12 months, and see where we can make improvements.

A friend just forwarded to me an article called “The Outsourced Brain” over at the New York Times. A sample:

” Until that moment, I had thought that the magic of the information age was that it allowed us to know more, but then I realized the magic of the information age is that it allows us to know less. It provides us with external cognitive servants — silicon memory systems, collaborative online filters, consumer preference algorithms and networked knowledge. We can burden these servants and liberate ourselves.

Musical taste? I have externalized it. Now I just log on to iTunes and it tells me what I like.”

This is going to gradually become a debate over the next few years as we pass more and more of our thinking and life over to algorithms. Stroustrup once said “Software runs civilisation”. I think we’re approaching the point where we can say “Software runs civilians”. There are obviously issues with this that need to be explored.

About a year ago I developed a hypothesis of what humanity would broadly look like 100 years from now. Some friends found my synopsis of this vision a little ridiculous: “You know the borg in Star Trek? That’ll be us”.

What I mean is that we are slowly moving our thinking out into the cloud and acting as one. Individualism is being lost, group-think is being encouraged. If that sounds a little Orwellian, can I just point out that we’re the ones encouraging it on ourselves - from CCTV cameras to collaborative filtering on Amazon - it is not being imposed on us.

The irony is that for all the menace of Borg assimilation and Orwellian dystopia in fiction, we are shaping parts of our society into something that mimics it in the hope it will lead to peace and harmony within society. Maybe it will, I don’t know.

The hope we have is that if we spend less time thinking about what music to buy, which directions to use to get somewhere, and trying to remember things we can get out of Google and Wikipedia anyway, we’ll have time for more important things. The question is what things are we doing with that time? Are we just filling that brain capacity with other trivia we don’t need?

Not for the first time, I feel that those of us styled “Software Engineers” have a responsibility to ask some questions here.

Let’s take a couple of ideas driving Innovation in the software arena right now:

• Work should be more like play
• 3D alternate Worlds are useful in some way
• People are finding it difficult to deal with the incoming flow of information

Each on their own can lead to ideas like Amazon’s Mechanical Turk, Second Life or better Bayesian filtering. In short, when you focus on an idea you can find ways of making software better. Some people ask themselves, wouldn’t it be great if you mixed some of them up? Say a 3D World where you work? Or handling information flows like a game?

What happens when you try and mix all three up? Well, I concur with TechCrunch when I say this example is just pure evil.

Yes, I can’t quite believe it myself. And I thought the ads in GMail would harm productivity in a mail application…

Tim O’Reilly is calling for Facebook to share social graph data so that systems can leverage all the data you’ve shoved into Facebook and use it within their own apps.

There are a couple of issues here.

Firstly, Facebook isn’t actually stopping 3rd-party API developers from knowing who your friends are, and if your friends agree to add an application, the app provider can see their graph too. What isn’t agreed yet is whether this should be made more open, or whether there needs to be a standard way of describing this data. There are all sorts of reasons why I might not want my “social graph” to be made available in an easily-manageable format, not least because it raises privacy concerns.

There is also the fact that Facebook’s business model relies on not making this data available. The “expose your data, and they will come” argument relies on a simple metric of conversion.

Within a company like Amazon, exposing the product catalogue by API is a no-brainer. The more places their stock list is available, the more chances they have of getting somebody into the system, the more likely they are to convert them into a sale. The porous membrane an API gives an app developer in this instance means 3rd-prty developers do the hard work of getting stock shifted in countless innovative ways the original company wouldn’t have thought of.

Facebook however, is different. The ‘conversion’ in their instance is getting somebody to look at pages with adverts on it. What they need is for their users to actively recruit more users - invite them inside the walled garden - and then try and keep them there. They’ve out-sourced the “retaining” part of the equation to developers (playing games, taking quizzes, sharing links, glorified e-mail), but by allowing their most valuable asset to be easily exported they are reducing their customer’s incentive to stay within the walled garden.

As always, it comes down to whether you have a right to that data, and whether you have a right to move it. I’d argue you do, but I’m suggesting it’s going to be hard for Facebook to allow you to take it wherever you want.

[UPDATE]: I realised there is a way to do this without Facebook’s permission. I’ve written it up on the site.

Apple Insider is giving us a thorough run-down on the history of the Virtual Desktop in anticipation of Apple’s launch of OS X Leopard. Leopard will ship with ‘Spaces’, Apple’s own take on the Virtual Desktop.

What has surprised me most in recent years is that nobody seems interested in why we continue to try and fit everything into the ‘Desktop’ metaphor at all.

As I understand it, when they were trying to decide an interface at Xerox PARC, they thought about who would use the tools they were developing. It is a sign of the times that they thought the only person likely to be sat in front of a keyboard any time in the future would be a secretary - so the developers went down to the typing pool and looked at how a typist might understand the World of work.

It’s for that reason, and that reason alone, we have the metaphor of a ‘file’ and a ‘folder’ and a ‘desktop’ within modern user interfaces.

There have been experimental interfaces to shift the metaphor, but these experiments have normally failed because the chosen metaphor has been so weak. Surely there has to be a better way to think about the way we work now we have fully embraced the hypertext document, the network as the computer and the social network?

A few years from now, people new to the Internet will find it impossible to imagine an era before video online. What use is that content though, if your user is visually impaired? How do we go about indexing and searching it?

A few weeks ago IBM announced an “Accessibility Internet Browser for Multimedia” over at alphaWorks. It addresses the short-comings of JAWS or voice-enabled browsers, and is built on top to of the Eclipse Rich Client Platform as a stand-alone application. It means the play/pause buttons (normally unavailable to visually impaired users) are stripped out and made accessible, and that playback doesn’t ‘clash’ with voice-synthesised browser operation.

Developers can add metadata to reorganise or simplify the content, provide additional information, add special navigation or even provide audio descriptions for movies using XML metadata.

It’s this last point that I think is going to be interesting. If this tool - or one like it - becomes standard, and video producers are encouraged to mark-up their content properly, existing search and relationship algorithms can be applied to video content. Right now searching video content is pretty limited - maybe by making it accessible, we all benefit.

Nat Torkington writes about the effects data flows are having on him. I also find that once every 2-3 months I suddenly notice there are many sources of information I’m not really reading any more, and so I have a purge.

He makes a point about Facebook which I find interesting. With some customisation, it is possible to tailor what you see and what you don’t (yes, you can turn off the stuff you’re not interested in). As such, we’re able to tailor how we spend our time on Facebook to make it useful for us. That doesn’t mean we spend any more time on Facebook, it just makes that time more productive.

What if all applications were able to work like that? What if we took the “News Feed” concept of Facebook and dropped it into the front page of all our applications and then allowed the user to specify what they wanted to see more or less of, and allow them to tailor the productivity?

My good friend and ex-colleague from way, way back, Andy Stothard, is currently on holiday in Vancouver. He loves the place and last time he came back he raved about it.

He’s just posted an article that struck me as an interesting take on file sharing. He’s sat down in front of his laptop, found a nearby computer sharing iTunes and decided to have a listen. It may be that “Gareth” didn’t intend for that to happen, or maybe he did.

What fascinates me about this is that there is an added context to the file sharing given by geography. It’s only people on the same immediate network you can see, and if Andy had really wanted to, he could have found Gareth and had a chat about his music collection (and maybe IT security) whilst he was there. What if it was “Gillian” instead of “Gareth” and Andy had really liked her taste in music. And she had found his iTunes collection to be interesting as well. And they’d made an effort to find each other?

File sharing has been criticised because it allows for an anonymous, amorphous mass on opposite sides of the planet to steal copies of music easily. What Andy was engaged in there didn’t take anything away from the publishers (he didn’t have a copy of the music, if he wants a copy he’ll need to buy one himself), added to his sense of the people around him in a foreign city, and potentially could have allowed for interesting conversation to break out between two previously unconnected people.

There is a lot to think about in the future with relation to the economics of creative works and the rise of ubiquitous digital access, but we need to realise that it’s not “File Sharing” that is the problem but “File Copying”.

Brian MCallister argues quite convincingly that “Popularity, in technology, is shit. Seriously.”

He has a point. When we aim to make something popular we are doing so for reasons of ego, and therefore attempt to compromise what it is we’re trying to achieve. We can’t do complex and useful to niché audience if we’re worried about being popular.

One version of this internal corruption of objectives is sometimes known as the “What would your mother think?” test in development. Would your (presumably technically illiterate, possibly senile) mother make of the gizmo you’ve just made? If the answer is “she wouldn’t understand it” then the trend is to simplify and to make things better.

But your Mum probably doesn’t care about your widget. What’s more useful is whether the people who are going to use it can. And that’s why, so the argument goes, that commons-based peer collaboration might be a better design practice than what we currently do.

It’s also why I think the future of innovation in software is going to be governed by companies making money whilst putting the source code out into the open. They quietly execute, iterating out improvements, making things better with each step, and then eventually the larger market catches on. The market catches on quicker if the source code is out there, resulting in better revenue streams.

Alas, we’re still in an age where the “intellectual property” myth still permeates our society, and trying to produce popular software seems more important than producing useful software. Sometimes it’s like the last decade was a dream…

This weekend just gone, I attempted to compete in Railsrumble 2007 with an application I call ‘Kagtum’. The idea of Rails Rumble is to take 48 hours to build a Rails app from scratch in a competitive scenario. At around 7:30pm BST last night I realised I wasn’t going to finish the app and so I asked to be withdrawn, as I explained:

Maybe it was the fact I decided to try and compete on my own, and therefore didn’t have the advantage of a team. Maybe it was the fact it took half the first day just to get a working application stack up on linode. Maybe it was the Saturday afternoon spent in the company of friends rather than coding. Maybe the idea was too ambitious.

Maybe I just wasn’t good enough.

Whatever it was, I didn’t get enough of my idea implemented on the weekend of the 8th and 9th September 2007, that I wanted other people to look at it. I failed. There is no app here.

The ideas I played with in those 48 hours though, intrigue me, and they will be worked on over the coming weeks and months. The end goal is going to either be very interesting, or an exercise in futility. If you want to find out which, you can keep an eye on the blog and I’ll be making announcements there.

I will be judging, and I look forward to seeing other apps, so good luck. Until next year…

The first day did kill me - linode was under heavy load (not surprisingly, with over 100 teams trying to get their application stacks set up) and the guidance we had been given by way of a screencast was inaccurate in places. Top tip: when you’re in a hurry leave the rdoc behind and always pass “-d” to gem install.

Anyway, I still hope to judge - and I’d advise anybody with an interest in innovation to look out for the announcement that you can sign up for judging and take a look at the apps that were finished - but I thought I’d talk about Kagtum a little bit here, because the core is almost done and I’m confident I can get a working app out of the door soon. I’m also tempted to open source it.

It all started about 2 years ago when I was left distinctly underwhelmed by Wikinews. The problems with wikinews are many and pretty obvious to anybody who spends a few minutes digging.

The primary problem to my mind is that they’re using a piece of software designed to build an encyclopedia to build a news website which means all articles are given equal footing. It seems reasonable that they should be given equal footing, until you realise that unlike an encyclopedia, not all news items are equal. A world-famous opera singer dying is not equal to a drunken brawl in my local town centre, and neither are equal to the Iraqi PM losing the confidence of the Iraqi citizenship.

However, the core idea - news written by, and for, everybody is a great idea. I’ve spent the last two years playing with lots of ideas in my head and watched emerging developments in the online news and journalism scene before I came up with an answer: quite simply it comes down to targeting relevance.

If I am in Manchester UK, there are stories that are local to me I’m interested in that somebody in New York doesn’t want to see. Likewise, there are stories happening on the other side of the planet which are important to me because they have an impact on me, or because they are in an area I have an interest in. The “perfect” news website will know this, and present just the articles I need to see. Ideally, I also don’t want to be bogged down with partisan and opinionated pieces - I want impartial, simple, Economist-news-page-style articles that give me the leader and then show me what is out there being written about it.

Thus was born the concept of Kagtum - the phrase “kag tum” means “to bring news” in Sumerian, the script of which is the oldest written language currently known to mankind. Kagtum will be a wiki news site that helps target articles based on relevance to you and your life. Relevance is everything.

The idea is quite simple, but the algorithm needs some polish before I can roll it out: we create a news story perhaps based on a report in MSM, or perhaps as a first-hand eyewitness account, that points to online sources if available. We then attach to that story some “impact profiles” based on location or a tags.

For example, a story happening on my street (say, planning permission for a new development) would have a geocoded location and an impact radius of the local neighbourhood. A story happening in 10 Downing Street would also be tagged with that location but could have an impact on the whole of the UK. Suppose the latter story was a policy announcement on Iraq - we’d add Iraq as a location impact as well.

I then login and give my location as my postcode or street, which is geocoded, and the software knows that the story in my street is relevant to me, so is the story in Downing Street. It knows therefore, what is relevant to each and every user and displays the appropriate stories.

Let’s suppose however I have no interest in Iraq. We can tag stories and users can also add tags to their profile that they’re very interested in or very disinterested in. If I said I wanted all stories marked “Iraq” to be pushed down the queue then its relevance to me would be lowered - it might still appear, just not as prominently.

In theory then, when I log in to kagtum, I would see stories about technology, politics and cricket, particularly with stories about my local neighbourhood (stories about technology in my neighbourhood would be even more prominent), whilst my friend who doesn’t care about anything but beer and football will see something perfectly tailored for his interests.

It may also be the case that there are multiple profiles for each user (home vs. work) and that a user can add multiple locations - where they live, where they work, where they grew up, where their parents live - and sees a mixture of stories about places they care about.

The biggest problem I had this weekend was developing the specifics of knowing which stories to show to each user. The problem isn’t hard algorithmically, but providing a technique that doesn’t harm performance and can scale to more than a few hundred users online at a time is proving a little tricky using standard ActiveRecord associations and using the methods baked into GeoKit by default.

There is also an issue of what we mean by “radius”. Saying “this story is important to everybody within 5 miles” is simple enough, but what if I say “to everybody within Greater Manchester”? I somehow need to know if a given longitude and latitude is within that district or not. The Radii Problem (as I came to call it whilst muttering to myself) is important and it’s difficult. I discovered it as I added a story in Washington that was important for the whole of the US - if I added a simple radius of 3,500 miles (to take in California) it of course also covered a huge chunk of Canada, Middle America, the Caribbean, the whole of the North Atlantic (including Ireland!) and most of the North Pole. For a story about domestic US politics, this is obviously needlessly “grabby”.

I have ideas on how to solve this problem, but they’re going to take a few weeks of playing with datasets from the UN and other agencies to be able to get them working smoothly.

There are other aspects I have planned for the site around developing narratives and helping individuals become kagtum journalists, but I’ll keep discussion of those for after the roll-out of both kagtum and the new vagueware.com.

I’ve turned comments on for this article, so if you have thoughts, ideas or suggestions, please leave them.

Whilst social photo sites like Flickr have grown over the last few years, many developers have been asking “now we have the resource, what do we do with it”. The first interesting application was tag clouds - it allowed you to see from the metadata in the system what was there and get an idea of how ‘heavy’ some concepts were. But what to do with the imaging itself? All those photos, and no easy way of making use of all the data they contained.

I’ve been ploughing through TED talks (no surprise that my favourite section is the What’s Next in Tech area), and have been meaning to post up lots of the talks, but one being discussed on a mailing list I’m on this morning is the demo of Photo Synth. Here’s the official video:

There are a couple of interesting things about this. Firstly, whilst the first half of the demo - a demo of Sea Dragon, a resolution independent image library - is interesting, there’s nothing truly novel about it. The only limitation stopping that system from being produced in the past is processing power. Every Computer Science/Software Engineering undergrad I knew had that idea whilst in the labs at University.

The second half though - the demo of Photo Synth - is what really grabs people’s attention. By computing vanishing points and common overlaps in images, it becomes possible to build a 3D representation of the object being photographed (in this case, Notre Dame) that you can take a virtual tour through. The applications are fascinating, not least because it takes mapping to a whole new level, and starts answering questions about what we’re going to start doing with all this social media.

One thing I noticed about that application, is if you upload a photo into a set that this software is processing, the software has to ultimately work out where you were in relation to the object. If just one photo in the set is geo-tagged (and many camera phones coming onto the market have in-built PS), I can work out your precise location. Now, let’s suppose you go on a tour of Paris. You take lots of photos all over the city. I now know your location when you were taking each of them. What’s more, the image will have within its metadata the exact date and time. I can, from that, construct a complete trace of where you spent your day from morning until night, with GPS-accurate location data, even though you didn’t have a GPS unit on you. Intelligence agencies are going to love this stuff…

Another interesting thought, is how this is being called a “Microsoft technology”. It wasn’t developed at Microsoft - they bought it in, and have worked out how to bring it to market. Well, when I say bring it to market, I mean do what Microsoft always do: make it available for Windows machines, but pretend the rest of the World doesn’t exist - the tech preview doesn’t work on Linux and OS X at the moment. This is a typical “flat World mentatility” prevailing at Microsoft I hope they’re going to change soon.

Microsoft are buying in a lot of innovation at the moment. They know they have a shortfall in innovative thinking (that’s what happens when your revenue is made up of sales of operating system software you can’t radically change and Office software everybody hates), but they have a big pile of money in the bank. By buying up the ideas and then pushing it out there, Microsoft are getting a lot of credibility within the geek community, and hopefully the idea-hungry culture will start to infect the rest of the company. I suspect a lot of people at Microsoft got a slight kick in the stomach when they saw Surface all over the web last month, simply because it’s such a radical change in how Microsoft looks at itself and answers the question “what is it we do?”

I have absolutely no respect for Microsoft, its software, or its business practices - I genuinely hope that for the sake of humanity the OSS community gets their act together and puts them out of business - but I’m starting to warm to some of the ideas and their employees.

In an article in Harvard Business Review Johnathan Zittrain takes a look at the risks facing the Internet in the immediate and medium term and how best to tackle the issues.

It’s interesting to me that his argument can be boiled down like this:

1. Technologies that can be used unintentionally by others to build useful things (generative technologies) are great.

2. The PC is great because of this. So is the Internet.

3. Because they are so generative people are able to create technologies that are actually harmful.

4. In order to avoid harmful software being run on their PCs connected to the Internet, users may choose to use less generative technologies, and move to appliance-like systems or services (e.g. iPhone, or only ever staying within Google/Yahoo sites)

5. We need to stop them adopting appliance-like systems in order to keep the Internet strong.

He goes about spelling that out in a few thousand words, but his ideas on how to do all this are a little light and tucked up at the end almost like an after-thought. It’s ideas though that I’m most interested in here, so here they are:

• Netizenship

  He suggests that all code should be subject to a Wikipedia-like editorial process where self-chosen experts vouch for whether the code you download and run on your system is harmful or not.

  This sounds novel, but I’m concerned about how such a system could be gamed by nefarious types. It may be that you would need to establish a trust network with certain users, but this would require you to expose information about the software you’re running: giving up your privacy to people who can name you is considered a greater threat by many than giving it up to criminals.

• Virtual Machines

  You download software and you put it in a special “container” on your machine where it can destroy what it wants to: it can’t get out to destroy your precious data. Once it has been running for a while and you trust it more and more, you can move it over into the “special” area where it has more rights.

  One question: why would I ever want to run the software outside of the container? Why not just provide a proper security model for all applications running on the system and take it from there? Unix sorted this one out better than Microsoft ever has in the 1970s, and with jailed/container environments from the BSDs and Solaris likely to hit the desktop Unix market within a couple of years, this might have some traction.

• More help from ISPs

  Zittrain argues here that ISPs turn a blind eye to zombie machines because they don’t want to help their users. This is completely and totally wrong.

  About a year ago I was in the ops center of a major multi-national ISP, sat with their abuse team watching how they worked. I was being considered for a contract to help them automate their processes (a contract cancelled as it was announced the ops team was to be relocated down South shortly afterwards).

  One of the jobs they took on was handling the automatic e-mails sent to them by AOL informing them of spam received by a machine on their network. They would then cross-correlate the IP address to customer records by hand, and give the customer a call and talk them through de-zombie-fying the machine. This *is* industry standard practice. ISPs who didn’t do this would eventually just find themselves being disconnected by their peers as they became more and more of a harbour for zombie machines.

  What would be better is more ways for ISPs to handle this automatically and to establish better trust frameworks. I also think that ISPs could relatively trivially prevent the most common spyware passing through their transparent reverse proxy cache boxes without causing major damage - the problem would be whether such action would be legal.

  I also think one area that ISPs could improve is education: a lot of people are pretty naive when they first get online and it’s only once the real story is told (or experienced first-hand) that users wise up. I think that at current rates enough people will be clued up enough to stop downloading spyware within a decade, but that’s a dangerous prediction, and there’s nothing stopping ISPs getting into the mix.

• Network neutrality for mashups

  This is an idea that sounds simple in practice, but actually defeats the point of generative technologies being adaptable, flexible, unregulated and novel.

  The premise is simply that if you write an application for a an API, your application is locked into that API’s vendor. They could withdraw service completely or just for your API key, or they could decide that they want to change the API at any time thereby breaking your application. In other words, if you invest time in developing to Google’s code base, you are suddenly beholden to Google: you can’t switch to Yahoo! or Microsoft APIs instead. This appliance-like style of development suddenly makes the “generative technology” of the Internet look quite stale compared to say the PC revolution.

  Zittrain argues that certain basic functions should be standardised so that once you write the application, it can work with similar services from other providers. If you write a Google maps mashup, changing it to work with Yahoo! should be as simple as changing a URL and an API key reference.

  It sounds simple, and standardisation is sure to come about over the next five years, but it’s actually a major undertaking. It would require all the major players to agree a baseline set of functionality, and that’s unlikely to happen with a group of commercial players.

  Where standards have arisen on the Internet - the IETF or the W3C - it has been a group of people acting in the spirit of non-commercial interest. Sure, the IETF meetings are dominated by Cisco, Juniper, Nortel and the ilk, but they are adopting a process first initiated by academics and amateurs: they’re second to the party, and are kept in their place by the tradition of open collaboration the IETF encourages.

  To create a standards body for web APIs now would require open source developers to somehow subvert the progress made by the commercial guys, or for the companies to stop behaving the way they have done for the last decade or so. It is an onerous task.

I think the error Zittrain is making here is that he believes these are problems that need to be solved in a way that is 100% efficient, or near 100% efficient. He also seems to be forgetting the human factors.

In terms of protecting users from harmful software, I believe this problem will go away within the next five years. It’s a dangerous prediction, but I believe that once users become more aware of the risks of software they are more likely to question where they source it from. It could be argued that without some kind of trust metric start-ups might find it hard to get traction, but start-ups have always benefited from the attention of early adopters who are clued up enough to know how to assess the risks of software like this.

In other words: if it’s getting good blog press, it’s OK to download. Most companies have woken up to the fact that you can lose every shred of credibility on the Internet within a day or less now, and credible company are not likely to push code that is going to harm your machine. By 2012, companies prepared to compromise their credibility will likely have no future. Zittrain’s assumption is a little like that of Cold War game theory: everybody is out to harm you, and the only right course of action is trust with verification, or outright suspicion.

As for the issue around APIs, again I think Zittrain is missing the human angle here. Yes, Google or Yahoo! could shut down their APIs at any point in time. Google have in fact stopped supporting an API or two in the past, but were careful in how they did so: they just stopped taking on new users for that API. If they were to leave developers high and dry, developers would feel the trust they had in the company had been broken, and they would make the effort to switch. Switching might be a pain, but it’s not impossible: most mashup apps out there right now are almost toy-like in their simplicity and can be recoded to a new API within a few hours. Once that happens, Google may as well drop all their APIs, and they know that.

I think eventually some form of standardisation is going to have to emerge, but all I can say about that with any degree of certainty is that it is unlikely to come from the commerical players.

In the middle of my RSS feeds this morning was a link to a wiki that - if it works - could develop into an incredibly useful tool.

Chris Heilmann (author of Beginning Javascript has set up the Business Case for Web Standards wiki because “there are a lot of presentations written about it but all differ in approach and content and collating all these great ideas can help us form a solid approach to selling web standards to the business.”

It’s obviously just starting out but it’s not too shabby already and should give most people a few pointers in the right direction. I particularly like the “Counterarguments” page where several of the arguments you’ll hear from managers and clients against adopting web standards are listed.

Don’t forget: it’s a wiki. If you think there should be something on there that isn’t, you know what to do, right?