Innovation in Software

About two years ago, job adverts started appearing in the industry that had interesting phrases in them: Ruby on Rails Rockstar; Ajax Master; Software Architect Ninja.

Eyebrows were raised. Was this just another example of the industry’s machismo playing out? Were we witnessing the next generation of kids coming up through the industry and rejecting the corporate ties of traditional job titles? Did HR really expect their latest technology recruit to play to stadiums, or attack enemies silently in the middle of the night?

What these companies were actually trying to say was “we need somebody self-motivated, dedicated to their craft, and who can take a lead in front of our audience (composed of customers, investors and employees)”.

Alas, the whole meme was misunderstood, those recruited into such roles were ridiculed, and adverts stopped trying to feel like they were recruiting heroes from Manga comics or NME.

However, we still need that mindset all over this industry. I’d go as far as to say it needs to become the default. We just need to cut out the hyperness and stop drenching the industry in irrelevant masculinity.

Let me explain what I mean by “that mindset” in more detail. Most of you will be familiar, but I think it’s worth recapping:

• Self-motived

  Anybody can learn how to write code, or fix a server. The difference between the majority of people in the industry today and the ones we want to see more of is how they learn, not what they learn. Some years ago I was working in a public sector gig where I had a couple of different people working under me. The difference between how they thought about their work was considerable. If I asked the first person to do something, the response might be “I’ve never done that before, so I’m going to need a few minutes to work it out, Google around, work out what I’m doing, spot the drawbacks, OK?”. Brilliant. The other guy would simply stop at “I’ve not been trained to do that. You need to send me on a training course”.

  There is no room in this industry for people with the second mindset any more. Rockstars don’t normally go to music school. Yes, training courses can be valuable, but unless you’re the kind of person who teaches themselves constantly, you’re not suited to this industry.

• Dedicated to their craft

  A craft you say? Better believe it. Most of us are taking a blank piece of framework and sculpting it into a finished product. “Rockstars” care about aesthetics, tools, even to some extent materials (or at least in the Ruby world we do). Software people should have ideals of craftsmanship, even when they’re pretending to be engineers.

  However, not everybody is dedicated to their craft. Many will not go to a conference unless somebody else is paying. Most do not re-factor their code to make it better in every way possible. 98% will never attempt to help other developers improve their code without an assumption of compensation. The really great people in this sector have the mannerisms and dedication of the minority, mannerisms we need to see become the norm.

  I’m not suggesting hair-shirts or years of religious solitude here, just an interest in the industry that seeps a little bit from time to time out of normal work hours and uses up some of their spare time and cash because the interest is genuine.

• Taking a lead

  I’ve met and worked with many “back-room mindset” developers. I once worked in a “pit” of developers and engineers at a household-name firm where team members were told to go home, have a bath, and come back tomorrow with clean clothes without any holes in them. To the uninitiated, this sounds disgusting (and to be fair, the pit did have a ‘funkiness’ about it), but in years gone by this was what small development teams were like. Guys who were in the back room, never asked for an opinion, never brought into management meetings, and who liked it that way. An invitation to a meeting was often met with a sneer by many of the people I worked with back in the late 1990s.

  Those attitudes are now (thankfully) disappearing. If you are the kind of developer who never wants to talk to the customer or management, you are useless in this industry. You’re dead wood. Sorry, but you need to realise that 80% of development is about communication, and good developers are the ones who lead that communication. Taking the lead is what we do. We are sculpting the future of organisations and job roles, we don’t get to sit in the cupboard and meekly accept specification documents handed down to us by non-developers any more.

There are more traits out there that we need to collectively foster, each of us at a personal level. This is just a starting point, this is what I thought of when people mentioned “Rockstar” developers. We just need a new name for it, something that doesn’t have all the connotations of “rockstar”, “ninja” or “master”. “Crafter” could work, but is too folksy for most people.

If the industry doesn’t start seeing these traits more in the kids coming up out of college, we’re going to struggle. The massive teams of old are dead. Small, focused development teams filled with people who think about their work the right way are the future. How do we foster these people?

Michael Arrington appears to be drunk. Or at least, he’s suggesting he is the kind of person who would wear a Life Recorder, which my casual talking-to-people-down-the-pub research about a year ago shows only drunk people seem to think is a good idea. Arrington though thinks they’re the wristwatch of the 21st century:

Imagine a small device that you wear on a necklace that takes photos every few seconds of whatever is around you, and records sound all day long. It has GPS and the ability to wirelessly upload the data to the cloud, where everything is date/time and geo stamped and the sound files are automatically transcribed and indexed. Photos of people, of course, would be automatically identified and tagged as well.Imagine an entire lifetime recorded and searchable. Imagine if you could scroll and search through the lives of your ancestors.

Would you wear that device? I think I would. [...]

Lost. The. Plot.

In his poll, just under a thousand people agree, but they haven’t thought this through. The people you meet may not wish your conversations to be recorded and many may not want to be photographed. You will not have the time to spend browsing through what happened last week because you have this week to live. They will be an extra device to manage, charge, and maintain. They will record things you will want to forget. They will mess up your social life as people actively reject and or object to your presence. They will mark you out as a self-absorbed prick.

Yes, there will be startups touting them and selling quite a few. I’m sure some people will buy them and annoy their friends with them. The narcissistic types in society will love them. A few celebrities might even buy them and have them regularly throw snippets to twitter. But they will be ultimately be considered a fad as people realise that they are more of an intrusion into their lives than an assistant.

Within military circles they might prove popular, and the police have been trialling “helmet cams” for years. Beyond those niches though, these things are an absolute waste of time and money.

Who on earth else is going to want these things beyond the self-obsessed?

It seems I’m not the only person slightly bemused by the reaction to Steve Jobs’ revelation he had a liver transplant.

Other people’s reactions include:

• He owes me more work because I’m a fan of his work
• My share portfolio is at risk because of this
• I want more shiny plastic things from Apple. This worries me they won’t be as shiny.
• He didn’t tell us something personal that we deserve to know

My reaction:

• You just had a fricking liver transplant? Wow, get well soon and don’t listen to those guys baying for your attention, you need rest, ‘k?

This isn’t about shares or gadgets or what he owes you. It’s about somebody who is seriously ill taking some time out to make sure they can live a little longer. You know “life”, that thing you take for granted? The thing that isn’t really about accumulating possessions but being able to breathe, eat, love, dream? The mob doesn’t get that – perhaps because it isn’t available to download in the App Store or listed on the Nasdaq…

And yet, somehow, this reaction is predictable. Much in the same way that sexism is alive and well in the industry, selfishness – in particular consumerist self-absorption – is rife. We are the pinnacle of consumerism. We thrive on early adopters, so we grow them. And what we grow, we reap – this is another problem we need to think about.

As a collective the consumers seem no longer to see the humanity behind technology, choosing instead to become voyeuristic onanists viewing technology almost in the form of a fetish they are addicted to. In fact, porn is a good metaphor for where we are right now: dehumanise and objectify humanity to serve a selfish need. It doesn’t matter if the lens the fetish is viewed through is that of a camera or the blogosphere, providing it’s possible to sit at screens satisfying our cravings in private.

Maybe that’s just the Catholic in me talking. The Agnostic in me thinks we can do better too, though.

Some will argue this is just the fruit of modern capitalism. All advertising in a capitalist free-market society relies on a principle of false idolatry, designed to invoke a sense of inferiority in our subconsciousness. Apple does it better than anybody else on Earth, taking their marketing cues from designer label brands.

I think we might have gone a step too far. We might need to dial it back a notch or three and re-imagine what we’re here to do. There is something pure about what we do that is beyond the gadget and the price tag, the plastic or the electronics. As Dijkstra said (and is quoted as saying on my business cards): Computer Science is no more about computers than astronomy is about telescopes.

Yet here we are complaining about the telescope manufacturer needing a liver transplant because all we want is more telescopes. Good. Grief.

We should be grateful for the genius behind every design decision that comes out of our great technology companies, and they should rightly be rewarded with praise – they advance society one little increment at a time. But when somebody takes time out to have a life-saving operation, there is something distasteful about a swarm of self-interested parties demanding to know where their share of the grief is going to come from.

I don’t know the answer, I don’t have the solution. All I know is that I don’t want to be part of the problem.

P.S. writing this story I was reminded of the most human thing I ever read by Jobs’: his commencement speech at Stanford in 2005 which I highly recommend taking the time to read.

The second article of the day discussing firms few have heard of outside of the industry. Bad habit. Sorry.

VMware makes some pretty interesting technology. Initially their product was as simple as allowing multiple virtual machines to operate on one physical computer. Then came the ability to dynamically move a virtual machine from one physical computer to another, seamlessly. That elegant addition to virtualisation allowed IT managers to condense application servers onto less physical hardware, reducing costs.

Now they’ve spotted that cloud/on-demand services are a possible future and have unveiled a cloud operating system that allows for, quote, “the foundation for internal and external clouds, using federation and standards to bridge cloud infrastructures”.

In other words, sometime next quarter you will be able to build out your data centre in such a way as to ensure that as demands increase, you can easily and seamlessly move some of the applications and services you are providing off into off-site data centres – even harnessing existing cloud infrastructures, if supported – so that you transparently respond to load conditions. You can have a smaller data centre for day-to-day operations, confident that if you need more processing power for a few hours, you can quickly and seamlessly hire it and pay for just what you use.

It’s a hard concept to get your head around, but once you do, you realise it is a powerful notion:

1. Here is my application stack consisting of web, application and database components
2. I am committed to this set of parameters in terms of SLA: do not let any of it fail; I need 99.999% uptime on the DB, 99.95% on the app; never move data outside of the EU
3. I’m off for a coffee now

If it works like that – and it’s a big “if” – it radically changes the way we think about building out infrastructure and using clouds. Right now I’m building three applications currently hard-coded to Amazon’s AWS due for launch within weeks. I’m now convinced I will need to refactor all of that code within six months. I also realise that the cool tools I’ve been building on to automatically respond to high-load periods are on the verge of being made redundant.

But there are downsides to vSphere. Big ones, too.

Firstly, see how they talk about “standards”? As Larry Dignan points out briefly in his analysis, there aren’t any standards. We’re in a whole new World here, and we’re starting from scratch. It could take 2-3 years for open protocols and standards to emerge with enough software being “cloud-aware” using them to make all this a reality.

Secondly, this is VMware, so make sure you bring money.

In fact, to get started with the “Standard” edition of vSphere you need to come to the party with $795 per CPU. That is not a typo. Got a quad-CPU machine you want to add to the cloud? Yup, you can get started for just $3,180. Got 200 of those machines in your data centre? That’ll be $636,000 please. What’s that you say? You need the “Enterprise Plus” edition for “Tier 1 applications”? Ah, that’s $3,495 per CPU, so for that data centre make your cheque out for just short of $2.8m please… thank goodness they aren’t pricing per core!

When VMware realise that the power of the cloud is with the small guy sat at home building an app that could scale beyond his wildest dreams but who needs to get started cheaply, that’s when this thing will take off. Until then, it’s just a beautiful idea at an ugly price point – and for the majority of web developers out there, that’s as good as making it a complete and total irrelevance. Shame.

I am truly amazed at the story the WSJ ran today suggesting that there are more people making money blogging than programming in America today. From the article:

When you think about it for a moment, this is insane. I don’t know how they are calculating their figure as there is no “blogger” row in the table they cite, but assuming it’s true It means that there are literally 452,000 people out there making a living (at least partially) from affiliate programmes, advertising and selling information products. Four hundred and fifty two thousand.

Maybe I’ve seen so many bubbles over the years that I’m now cynical of large numbers, that seems an awfully large number. What’s more, it’s no surprise that this content with its sensationalist, highly-segmented and tailored tone is taking readers away from newspapers. I’ve discussed before the problems of newspapers and I don’t wish to discuss them again just yet, but the WSJ does ask some interesting questions about this new breed of content producers:

“While many bloggers probably support unionization in general, they have no union of their own. Most have no benefits, yet they work long hours in front of computer screens which could cause a variety of health ailments. And the owners of the big sites most often pay their bloggers as freelancers, avoiding all of those taxes and benefits that newspapers have to pay for their writers.

For now, bloggers say they are overwhelmingly happy in their work, reporting high job satisfaction. But what happens if they, too, lose work; are they covered by unemployment insurance if tastes change and their sites go under? Are they considered journalists under shield laws? Are they subject to libel suits? Are there any limits to the opinions they churn out, or any standards to rein them in? Is there someone to complain to about false blogs or hidden conflicts? At the recent Consumer Electronics Show, Panasonic outfitted bloggers with free Panasonic equipment; did that affect their opinions about the companies they wrote about? There are more questions than answers about America’s Newest Profession.”

I feel that as newspapers submerge into the quagmire of their own making, the trained journalists they release will emerge on their own two feet and find a way to take some of the earnings pie out there for themselves, whilst also addressing these issues. There are plenty of models on the horizon for individuals or small teams of journalists to be able to produce the content we need, get paid well for it and in a work setting that protects them, and all without the flabby masses of managers who don’t understand this new model of the World.

Interesting times we live in, eh?

You heard then?

This is one of those deals where all and sundry outside of the industry ask “Who is buying who? What now?”, whilst those inside the industry faint with disbelief, anger or joy.

To the man in the street this story could be headlined “Supplier of over-priced RDBMS and related services purchases supplier of over-priced hardware and services”. Most people have never seen a Sun computer or had to install or manage an Oracle database. They might have heard of these companies, but don’t know what they do. Somebody once asked me if IBM was still trading – if IBM is thought of like that, imagine what this duo’s impact on the masses is.

Except every time you deal with your bank, your utility companies, pay by debit or credit card in a shop, use your mobile phone or apply for Government benefits, you are using Oracle and Sun technology somewhere along the way. Almost every single time.

The conspiracy theorists of the last 15 years have pointed their finger at Microsoft and screamed “monopoly” every time they could. Microsoft’s deployed technology is almost an irrelevance now. Everything from J2ME on your phone and MySQL behind your Wordpress site, all the way through to your HR department and bank IT systems are now all under one technology roof.

Is this a good thing? Is this something we should attempt to stop? Truth is, I really don’t know. Neither does anybody else. I seem to be one of the few people thinking about it like that.

Right now most people are worried that this will kill off MySQL but that isn’t going to happen. A few with heads in the sand argue this changes nothing. A very small body of people are just glad they can breathe easy now and wash their hands of a problem child.

In other words, people’s fears are likely unfounded, their denials are silly and a few people got out of a stock position they were glad to be out of.

And now all this technology is sat there under one management structure. What will they do with it? Will it flourish? Will it flounder? History suggests this is likely to be an awful deal for all concerned with innovation and a great deal for shareholders. Time will tell.

This morning I find that bastion of journalism Valleywag (please note sarcasm here), has decided to lay into Marissa Mayer the VP who oversees Google’s core search business. Their analysis of her comes down to this:

• She seeks publicity
• She has influence in the company at a senior level
• She dictates the style of web pages without a style guide
• She is a perfectionist over candidate selection
• She points to athletics as part of her life but her times aren’t great

And then… well, this:

“Did she really mean to invite media scrutiny of her athletic career? What’s really telling about it: In the handful of times where Mayer has competed on her own, without the backing of a billionaire ex-boyfriend and a pliant boss, she has proven to be an outright failure.

At the beginning of the piece, Mayer once again denies rumors of her impending departure from Google — rumors which Valleywag first reported. Perhaps she has realized that without Google, she’s nothing. Can you blame her for clinging to her job?”

The vindictive tone, the over-arching sense of misogynistic snobbery is just astonishing.

Let’s take another figure within the tech sector who is influential: Steve Jobs. He, for what it’s worth:

• Seeks publicity
• Has influence in the company at a senior level
• Dictates the style of some products, sometimes with a style guide, sometimes not
• Is a perfectionist over candidate selection

What about Bill Gates when he was at the helm at Microsoft? Well:

• He saught publicity
• He had complete influence in the company at a senior level
• He dictated the style of Microsoft software without a formal style guide (and it showed!)
• He was a perfectionist over candidate selection

Yet we call them symbols, heroes, geniuses even. Marissa may indeed have pointed to athletic engagement where her performance is not above-average, but is it wrong to aspire? Valleywag’s little dig by turning around Mayer’s words that “good students are good at all things” to mean she should be good at everything she does is frankly unfair.

She got picked on because – and only because – she is a woman.

I shouldn’t be surprised. This is a sector that struggles to attract and retain women. Given the tech sector now effectively runs Western civilisation, this is a problem beyond reasonable description. However, it seems that time and time again the women who get close to the top get torn down again.

Take Kathy Sierra whose articles on usability were perhaps amongst the best written in that area, well, ever. She was helping people understand how to improve software and services in real, tangible ways. Didn’t ask for a penny for it, just did the odd speaking gig and basked in the glory the majority of us held her in. Astounding writer. Anyway, you’ll never guess what happened? Well she got death threats and the occasional suggestion she should be raped. She shut the blog down and got out of Dodge. We as an industry are worse off as a result.

Gentlemen (and it is you who I speak to specifically): what the crap are we playing at here?

I believe in equality of the sort that means that women who genuinely screw up should be called for it as much as any man and should not be put in cotton-wool environments. But to pick out women for doing things exactly the same way her male peers do (or better) and then basically using the medium of the blog to bully them because they remind you of that cheerleader who wouldn’t go to the prom with you, is quite frankly despicable.

I not only want more women to be attracted to this sector, but I think as a society we need them. Not as social media analysts, but as real entrepreneurs, managers, engineers and designers. I was foolish enough to think the Sierra episode would be a watershed moment for us as an industry. The reality is we remain far away.

It seems Valleywag has found somebody else to bully and I have a horrid feeling this is going to get more miserable and vindictive before it gets better.

There is an awful lot of consternation going on amongst the suits behind online news websites.

The problem, it seems, is a colossal error made by most newspapers in offering you all their content without asking for some money from you. Their reasoning is that this forced them into the route of monetising their websites through advertising. This has never been profitable from the day they first did it. Now, they argue, the time has come to change all this and you must pay.

I want to briefly point out how this happened, why it’s the news organisations’ own stupid collective faults, why people will never pay for this content, and how business models online will need to evolve over the next few years. I pick “News” as a sector, simply because that’s where my head is right now but it applies to any other sector as much.

I also want to point out that whilst I contribute to a blog over at The Manchester Evening News this isn’t about the MEN. This is about a global industry that is in trouble. I don’t know if a single word of what is written below applies to the MEN, because I don’t work for them and don’t have enough experience of the inside of the organisation to know what happens there. Maybe some of the below does apply, but I’d be surprised if it all does.

Let us travel back in time to the early 1990s. Back then, 28.8K modems were considered speedy and there were but a handful of ISPs offering dial-up access in the UK – typically for about £10/month – to the Internet. The main applications were mail and Usenet. The web did not exist.

But one bright day, it did, and then there was light, etc. There was a rush to this new territory and a few keen early adopters started to think about what this might all mean to newspapers.

Strangely, the gentlemen of perhaps the UK’s then most-traditional newspaper, The Daily Telegraph, were the first to consider publishing their content online. They decided then that advertising was the way to support this venture and so the first UK newspaper website was launched with all manner of gaudy, flashing images helping to prop up its cause.

Why did they do that?

Well, let’s think about how newspaper management typically think about their business. To you or I a newspaper is a source of news. To a journalist it is a means of employment that allows keen types to investigate the World around them.

To newspaper owners, it’s an advertising hoarding they sell space in, and in order to get people to look at it, they shove some news stories in next to the main content in as inexpensive a manner as possible.

That might sound cynical, but reality often does.

When Rupert Murdoch’s advisers were trying to convince him to buy MySpace the argument that sold him was “you’ll be selling advertising next to content you didn’t have to pay for the production of. What gets better than that?”. That mindset is dominant in online news.

When you buy a newspaper (if you still do), your money is not to pay the wages of the staff who produced the paper – the money is to offset distribution costs mostly. The printing, shipping and selling of newspapers is an inconvenient but necessary cost and in order to keep everybody from the newsagent all the way through to the inky-fingered man who produced the paper happy, you pay your pennies for the finished article.

The main source of income always has been and always will be, the advertising. In case you’re wondering, it’s the same story in the magazine industry.

Now, let’s move things online. The distribution costs of a website are tiny compared to that of a physical newspaper. If you were to choose the comparatively costly hosting services of Amazon’s EC2 cloud resource, and had 10 small server instances running and were fortunate enough to be shifting a few hundred gigabytes of data a day, your total costs would be less than $60 a day. That’s about £40. Compare that to the £650m needed for the printing presses of the modern paper editions.

Therefore if distribution costs are near enough to zero for online editions, why bother asking the customer to pay for them? It’s almost sound logic, except they then made a major, major error.

They gave the advertising away for free.

Phone any regional newspaper title in the country and speak to their ad sales team. The conversation will result in them offering you a rate card for the print edition. Sound sceptical about the costs and benefits. They will offer space online for free. Every time. I know, because I’ve spoken to quite a few ad sales teams in the last year.

In essence to secure the advertising for the print edition, they have in the past completely undermined the business they need to survive in the future. They have told every one of their advertisers that online adverts are not worth paying for.

Let’s move forward and look at the state of business affairs within the news sector today.

All of a sudden, as if by magic, the clouds have moved apart and revealed that their website is a cost-centre. In order to keep up with the Jones’ they have had to build dedicated online news teams, larger server infrastructures, bring programmers onto the staff and throw money at their website in order to remain attractive to shareholders and other investors. Slowly the ad sales team realise that in fact the future of their business is online and they start to unpick their years of collective suicide, slowly trying to build up the revenues they have worked so hard to destroy.

It’s all a bit grim. One newspaper group dominant in a Southern part of the UK almost sounds desperate when you phone them and enquire about online sales. I genuinely feel sorry for them sometimes.

In short, none of the people responsible for picking up revenues really know what they’re doing any more. The World has changed around them whilst they have stood still, and it’s all too much. They are thinking in terms of traditional newspapers and traditional ad sales, and struggle to find the revenues they need to break even. Slowly, they realise they can’t fix this. They need people to blame. The usual people they wag their fingers at are:

• The BBC. They get a license fee, they operate the largest online news websites and hey, it’s just not fair, right? They should stop!
• The advertisers for not spending enough, mainly because for 15 years they’ve been told that online adverts are basically free
• The audience for demanding free online news like some sort of self-interested collective mob (which they are!)
• Anybody and everybody but themselves.

But wait, there’s hope. Amazon’s Kindle is being thought of in some quarters as the “news iPod”. The people behind the curtain at major newspapers groups all over the World are lining up to try and sell you subscriptions to their content via eBook readers like the Kindle (and perhaps the iPhone and other mobile platforms), and then slowly their digital operations will move to subscription-only content. News websites will slowly shut down and become adverts for the print and subscription-only content via the new technologies sure to become dominant in coming years.

Except of course, the audience won’t care. They won’t sign up. Why would they if even a few websites remain free and open for business? Nobody at Wikinews is busy trying to find a business model that works. I’m pretty certain the BBC won’t be allowed to. That’s enough to wreck this plan.

And yet, the news organisations lumber on hopeful something will change soon. No changes in business structure, no clear thinking about how to go forward, no understanding of online business models.

Here’s a good example of bad thinking about what it takes to succeed online. Craigslist has about a dozen employees and clears about $100m a year in revenues, thanks to their UGC structure. That’s some margin! The BBC gets £3 billion a year in license fees, so by the same gearing (ignoring the obvious differences in structure and intent of a non-commercial entity like the BBC) they shouldn’t need more than perhaps a few hundred journalists. Do you know how many journalists the BBC employs? 7,000. Seven. Thousand. And they, for their budget, are considered quite an efficient news outfit.

So, let’s summarise where they’ve gone wrong so far:

• Newspapers didn’t understand the web when they got started
• The ad sales teams on newspapers took too long to understand online business models
• They then undermined their own business model
• Newspapers realised they need to take a new mindset and blame everybody else for where they are right now
• They then decide the option is to try and get customers to pay for content they don’t want to pay for
• Absolutely no consideration is made of what being an online newspaper consists of

Not sounding good is it?

What to do? It’s OK me sat here mouthing off about how stupid they all are, but can I offer some insight? Well, here’s some thoughts to people running newspaper groups right now:

• Online news is not the one-way broadcast medium you’re used to. Use your audience. Wikinews is cheap to produce. Your newspaper could be. Kagtum will be.
• Online ads are not the same as print ads and can’t be sold in the same way. I don’t know of a single news website in the UK where I can buy PPC ads with my credit card right now.
• Even worse, online ads aren’t very profitable anyway. Google ads work because they enhance the product. How do you make money whilst enhancing your own content? Price comparison, auction and digital distribution models have figured this out well.
• Customers do not want to pay for anything. Ever. You have to work hard to convince them it’s worthwhile. You’re doing an awful job of that, so stop trying to do it
• Invest in journalists, programmers, graphic designers, information visualisation specialists and people who understand online content. Fire the majority of your middle management who aren’t directly involved in producing content or ad sales.
• People will always value investigative journalism
• Think laterally about revenue in an online medium

It might all sound a bit extreme, and I’m sure many will criticise me for my naive attitude, but I say these things because I care. We need good journalism. Wikinews might be cheap to produce, but it doesn’t allow for investigative journalism, an aspect of news reporting which I believe is critical to democracy. We need newspapers.

I’m writing this because I want to both signal where I’m thinking in terms of my own projects, and because I’m also fed up sifting through dozens of articles a day by everybody in the industry from the Western US to the East of Asia all blaming the dire position of news today on a lack of subscription models. I think there is a future for online news, but I suspect it will need the current generation of newspapers to go bust for the new models to successfully emerge.

What fascinates me about all this though, is that there are so many people whose future relies on online business models, and yet they don’t understand what that means operationally. Interesting times.

Over 2008 some remarkable technologies emerged quietly that seem to be gaining traction within the industry. Whilst around for years, I am confident their time is about to come proper.

If you were to ask most people on the street who the most innovative technology firm was of 2008, you are likely to see a familiar list: Google, Apple, maybe Microsoft or perhaps some of the smaller outfits that have crossed into the mainstream like Facebook or Twitter.

Few people will mention Amazon.

In fact, if you point out that Amazon is right now perhaps one of the most innovative technology firms on the planet, people will raise an eyebrow and ask “What? The online shop? Where I get my my books and DVDs from?”.

This typical reaction is perhaps caused by only industry participants having seen so far just how Amazon are disrupting the economics of doing business online. Now anybody can have access to infinite storage arrays, huge compute clouds, masses of humans to complete complex tasks and distribute content across the globe as fast as possible, all without a penny of capital expenditure: you pay only for what you use. You can even send your physical products to be stored in Amazon’s warehouse and they will for a fee handle order fulfilment for you, again only paying for what you use.

Capital expenditure is a start-up company’s biggest problem. Remove it and suddenly anything becomes possible in the start-up World. This is big. Very big.

The real beauty is perhaps the fact Amazon made this move originally to use their own infrastructure more efficiently. If Amazon has lots of spare compute capacity ready to serve pages during their busy season (the run up to Christmas), why not lease it out the rest of the year? And yet this strategy has started to offset their own infrastructure costs so much I wouldn’t be surprised if within a few years their operational costs tend towards zero. The most powerful e-commerce platform on the planet, and everybody else is paying for it for them.

This has caused people to sit up and notice. “The Cloud” is now the hottest buzz term in the industry and all players are trying to figure out a strategy to compete.

One of the issues is that Amazon’s infrastructure is not as simple to use as it could be. Plenty of firms see a gap to try and make things simpler: one of the biggest complaints about S3 is that you need to use custom APIs instead of open standards like SFTP or WebDAV; EC2 needs a more complex understanding of systems administration and data storage than traditional models; for many applications it’s overkill or too generic, and so on.

If you break the components needed for a web application into its constituent parts from platform and compute capability through to storage services, you realise that at each level there are numerous companies trying to find a place in this market from Google and Microsoft through to unknown start ups, some of whom are attempting to make access to other cloud services easier to use and therefore are some sort of “meta-cloud” service.

This is thought to be a paradigm shift in how developers think about developing and delivering applications, but what we have seen to date is likely just the tip of the iceberg. For a number of years now traditional engineering firms (notably Rolls Royce in particular), have realised substantial revenue growth comes not from product sales – competitors can easily counter advances in product development – but in services. It seems the computer industry is starting to cotton on, and companies like AMD are thinking about how to ride the Cloud hype into the services sector. Even Microsoft are considering versions of Windows that users pay for by the hour.

It is perhaps because these service revenues lock a user into a provider’s business that some point out the dangers but I believe in time we will allay such fears by changing how we describe, define and use cloud capacity: it’s perfectly possible for us to control our own data and rent storage and compute capability as we need it, perhaps without realising that is what we are doing, without surrendering our rights and privacy. It is not yet a trivial job to do so, but surely over 2009 we are likely to see services emerge that allow consumers to harness cloud concepts and capabilities without needing to understand the detail.

Ultimately though, the real benefit over the years ahead will be the possibilities these services offer the programmers wise enough to harness them: without the CAPEX requirements, the only limitation developers seem to encounter is that their imagination struggles to break free from the bonds that have dominated careers to date. It is hard after decades of worrying about RAM, storage and CPU limitations to have all them removed, or at a minimum re-shaped. This is the beginning – if we can struggle to imagine it – of something huge.

So whilst I’m thinking opportunity, it seems the incumbents in the web application industry are starting to separate the wheat from the chaff, or at least according to ReadWriteWeb, Google is at any rate.

I’m actually genuinely surprised at the announcement on Google Video simply because it’s a better service, despite being less popular.

If Yahoo! do this, expect it to become a usable service again.

Interesting times…