Innovation in Software

You heard then?

This is one of those deals where all and sundry outside of the industry ask “Who is buying who? What now?”, whilst those inside the industry faint with disbelief, anger or joy.

To the man in the street this story could be headlined “Supplier of over-priced RDBMS and related services purchases supplier of over-priced hardware and services”. Most people have never seen a Sun computer or had to install or manage an Oracle database. They might have heard of these companies, but don’t know what they do. Somebody once asked me if IBM was still trading – if IBM is thought of like that, imagine what this duo’s impact on the masses is.

Except every time you deal with your bank, your utility companies, pay by debit or credit card in a shop, use your mobile phone or apply for Government benefits, you are using Oracle and Sun technology somewhere along the way. Almost every single time.

The conspiracy theorists of the last 15 years have pointed their finger at Microsoft and screamed “monopoly” every time they could. Microsoft’s deployed technology is almost an irrelevance now. Everything from J2ME on your phone and MySQL behind your Wordpress site, all the way through to your HR department and bank IT systems are now all under one technology roof.

Is this a good thing? Is this something we should attempt to stop? Truth is, I really don’t know. Neither does anybody else. I seem to be one of the few people thinking about it like that.

Right now most people are worried that this will kill off MySQL but that isn’t going to happen. A few with heads in the sand argue this changes nothing. A very small body of people are just glad they can breathe easy now and wash their hands of a problem child.

In other words, people’s fears are likely unfounded, their denials are silly and a few people got out of a stock position they were glad to be out of.

And now all this technology is sat there under one management structure. What will they do with it? Will it flourish? Will it flounder? History suggests this is likely to be an awful deal for all concerned with innovation and a great deal for shareholders. Time will tell.

Over 2008 some remarkable technologies emerged quietly that seem to be gaining traction within the industry. Whilst around for years, I am confident their time is about to come proper.

If you were to ask most people on the street who the most innovative technology firm was of 2008, you are likely to see a familiar list: Google, Apple, maybe Microsoft or perhaps some of the smaller outfits that have crossed into the mainstream like Facebook or Twitter.

Few people will mention Amazon.

In fact, if you point out that Amazon is right now perhaps one of the most innovative technology firms on the planet, people will raise an eyebrow and ask “What? The online shop? Where I get my my books and DVDs from?”.

This typical reaction is perhaps caused by only industry participants having seen so far just how Amazon are disrupting the economics of doing business online. Now anybody can have access to infinite storage arrays, huge compute clouds, masses of humans to complete complex tasks and distribute content across the globe as fast as possible, all without a penny of capital expenditure: you pay only for what you use. You can even send your physical products to be stored in Amazon’s warehouse and they will for a fee handle order fulfilment for you, again only paying for what you use.

Capital expenditure is a start-up company’s biggest problem. Remove it and suddenly anything becomes possible in the start-up World. This is big. Very big.

The real beauty is perhaps the fact Amazon made this move originally to use their own infrastructure more efficiently. If Amazon has lots of spare compute capacity ready to serve pages during their busy season (the run up to Christmas), why not lease it out the rest of the year? And yet this strategy has started to offset their own infrastructure costs so much I wouldn’t be surprised if within a few years their operational costs tend towards zero. The most powerful e-commerce platform on the planet, and everybody else is paying for it for them.

This has caused people to sit up and notice. “The Cloud” is now the hottest buzz term in the industry and all players are trying to figure out a strategy to compete.

One of the issues is that Amazon’s infrastructure is not as simple to use as it could be. Plenty of firms see a gap to try and make things simpler: one of the biggest complaints about S3 is that you need to use custom APIs instead of open standards like SFTP or WebDAV; EC2 needs a more complex understanding of systems administration and data storage than traditional models; for many applications it’s overkill or too generic, and so on.

If you break the components needed for a web application into its constituent parts from platform and compute capability through to storage services, you realise that at each level there are numerous companies trying to find a place in this market from Google and Microsoft through to unknown start ups, some of whom are attempting to make access to other cloud services easier to use and therefore are some sort of “meta-cloud” service.

This is thought to be a paradigm shift in how developers think about developing and delivering applications, but what we have seen to date is likely just the tip of the iceberg. For a number of years now traditional engineering firms (notably Rolls Royce in particular), have realised substantial revenue growth comes not from product sales – competitors can easily counter advances in product development – but in services. It seems the computer industry is starting to cotton on, and companies like AMD are thinking about how to ride the Cloud hype into the services sector. Even Microsoft are considering versions of Windows that users pay for by the hour.

It is perhaps because these service revenues lock a user into a provider’s business that some point out the dangers but I believe in time we will allay such fears by changing how we describe, define and use cloud capacity: it’s perfectly possible for us to control our own data and rent storage and compute capability as we need it, perhaps without realising that is what we are doing, without surrendering our rights and privacy. It is not yet a trivial job to do so, but surely over 2009 we are likely to see services emerge that allow consumers to harness cloud concepts and capabilities without needing to understand the detail.

Ultimately though, the real benefit over the years ahead will be the possibilities these services offer the programmers wise enough to harness them: without the CAPEX requirements, the only limitation developers seem to encounter is that their imagination struggles to break free from the bonds that have dominated careers to date. It is hard after decades of worrying about RAM, storage and CPU limitations to have all them removed, or at a minimum re-shaped. This is the beginning – if we can struggle to imagine it – of something huge.

I’m almost human again after last week’s ‘grand tour’. I am however a little behind on blog articles. As promised though, some notes from Thursday’s mashup* event:

Sun Microsystems were providing space for this event at their office in Sale. It’s a great venue, but the trip out of central Manchester during rush hour left your correspondent a little frazzled. All trips out of Manchester leave this correspondent frazzled though, and it may have just been the task of crossing 8 lanes of heavy traffic that did it.

We opened with David Terrar from Blognation giving us a run down on where he’s seeing ‘Web 2.0’ technology in the enterprise. He touched on the BBC using blogs and wikis extensively, and discussed Pfizer’s use of the same. A lot of people seem to be trying to lock into consumer-orientated applications because the current success stories – Facebook, Flickr, MySpace – are in that arena. I think David is onto something in that the real money is bringing this style of social technology into the enterprise.

Simon Grice then did a more bi-directional talk on the premise that Web 2.0 doesn’t exist. I piped up and got agreement with Simon on the point that indeed the very first web browsers had “edit” capabilities and at some point, somebody decided it was too anarchic and turned the web into a broadcast medium. We’re now just getting back to where the web should have been more than a decade ago.

I also had to admit to the room that I have hired deviants in the past. Long story, I’ll fill you in some other time…

Simon discussed the disruptive nature of new media, and cited his own recently-launched testcard.tv – a site lawyers working on behalf of media companies will no doubt be particularly interested in.

UPDATE: Checking URLs, I notice that testcard.tv has been put into an “Under maintenance” mode, citing the takedown of a similar service and asking people to head over to their blog – it’s one way to get traffic I suppose. :-)

Next up, was Lee Strafford of Project Sahara. He’s trying to gauge interest on this side of the Pennines in what could be an interesting project. Talking to people after the meeting, the consensus was “wait and see where it goes”, which is unfortunate as it needs people to get involved now for it go anywhere. I’m going to do some talking around over the next few weeks and see what interest there is in getting stuck in at an early stage.

We then skipped the break – Ouch! Don’t do that again! My brain was hurting! – before moving onto case studies of NetVibes, edocr and Meecard. All interesting stuff, but I was already quite familiar with NetVibes and edocr – Meecard was a little food for thought though.

There then followed some discussions, some light networking, some pizza and a few beers before wandering off home.

All in all, this is a more polished and business-focused version of GeekUp. As a format it has a lot of potential, and providing the case studies and speakers can be lined up it could be a regularly anticipated event. Some people complained about the £25 entry fee (a sentiment I broadly agree with), but I know several people want to try and help out on that front.

I also find myself a little disappointed that I’m going to have to wait until February until the next one, so that has to be a reassuring thumbs-up for Manoj and Simon.

Johnathan Schwarz announced something yesterday that at first sounds quite dull until you read through his justification and where he’s heading with it.

Basically they take the UltraSPARC T2 blueprints, the core design files and test suites and then they release the whole lot under GPL online

It’s a gutsy move in that it completely breaks the model of what you’re meant to do with R&D expenditure in the computer industry. He already has experience of this with OpenSolaris of course, but to do this with hardware is unprecedented.

The impressive figures he quotes in his justification for opening up UltraSPARC also pricks my ears:

You’ll recall we followed this path with our software business – decoupling Solaris from its exclusive focus on Sun hardware. That experience validated the obvious: the market for Sun’s innovation is always larger outside of Sun, than inside. When we opened ourselves to the market, our business grew faster (Software grew 13%, year over year, faster than Sun overall). Now we’re following that path with our microelectronics business.

Let’s make this clear: he opened up Solaris open source, allowed it to run in more places, and the business grew 13% year on year.

It seems counter-intuitive: give something of value away, watch the value of your business grow. But that’s how software works as a commodity. License keys have no value, user base has value. Keeping a little team of ‘rockstar developers’ in-house to develop code has no value – having thousands of developers passing ideas through you does have value.

How you monetise that needs to change based on the area you’re sat within, but the argument that open source is not a valid business model increasingly seems to be looking to be a myth. Arguing the only way to make money is to charge for everything you do ignores the fact you can probably make a lot more by allowing other people to do it for you, on similar terms.

This is the coolest infrastructure project I have seen in a long, long time. Project Blackbox is Sun’s ‘data center in a shipping container’ and can come complete with two petabytes of storage, and provide enough processing power to match almost any corporate data centre I’ve ever heard of. What’s more, it’s energy-efficient, recyclable, and you can ship it to anywhere on the planet. The innovation of this thing is just… well, astonishing.

Using a standard 20-foot shipping container, what they provide is a fully racked, cabled and cooled data centre you just ship to wherever you want – basements, roof-tops, oil platforms, wherever. You hook up electric, networking and water (for cooling), and you suddenly have a completely working data centre. It has alarms, GPS tracking, and shock absorption for transportation and security. Once you’ve finished with it, you can either pick it up and ship it somewhere else, or if it has reached end of life, just ask for Sun to come and take it away for recycling and environmentally friendly disposal free of charge – a big factor now we’re in the age of the EU Waste Directive.

It can be populated with 250 Sun Fire servers configured for grid computing across seven racks inside the container, with an eighth rack containing network switches, dehumidifier, thermal management systems, alarm systems and EPO controls. Sun are keen to make things configurable though, and they suggest you could hold 120 Sun Fire T2000 systems, or 250 Sun Fire T1000 systems with 2,000 cores. They claim that 250 CoolThreads servers support four times the number of Web users and provide five times the efficiency of the same number of similarly configured Dell Xeon servers.

What’s more, they say customers will also save approximately $1,000 per CoolThreads system per year in energy costs over traditional datacenters, and customers in Northern California – which, despite being home to Silicon Valley has surprisingly few data centres – qualify for energy rebates of up to $240,000 if they upgrade from Xeon-based systems to Sun Fire CoolThreads servers in Project Blackbox.

Many years ago I used to be an engineer at an ISP that had for various reasons built its data centre in a converted cowshed in the middle of nowhere. One of my jobs each morning and each evening was to ‘sort out the cooling’. Because we had not had room to put proper aircon in and we had quite high rack density, we were wheeling around portable aircon units just to keep the temperature below 40 degrees celsius ambient and stop routers shutting down.

What I, and my colleagues, would have given for this box back then. I have been in about half a dozen data centres since, and none of them seem to be as well thought out, environmentally friendly, flexible or usable as this thing is.

The problem, of course, will be price. This box is not going to be cheap, and not surprisingly, they’ve not put up a sticker price anywhere on their dedicated website for it. I’d guess the price would be hovering around the $1 million mark, but in all honesty even if it cost twice as much it would still be arguably good value. I want one. Now. :-)

Sun is going to be giving full support to Ruby, and will be shipping a Ruby parser built on top of the Java Virtual Machine apparently with Java and Ruby both being fully supported by the VM.

I still am not sure if this isn’t some weirdo-World practical joke. I expect to wake tomorrow morning to find this is just another version of the Parrot Joke (which eventually became a real project).

If true, it means Ruby is about to get incredibly mainstream, and I’m going to spend the next five years having some real fun. It also means Joel is going to look even more stupid with regards to stating that Ruby is not ‘enterprise-ready’.

One of the biggest problems with Ruby right now is the current parser is slow and we’re having to jump through hoops just to get performance up. With web applications it’s not a major problem until you start to scale, and then most of your performance problems can be ironed out with some cacheing and database schema optomisation. Through more clock cycles at the database server, and most web apps get quicker – Ruby isn’t the slow part in that equation. However, this changes the game – _if_ the parser Sun ships is nice and fast.

The mailing lists however are strangely quiet. It may be that too many people want to adore Matz for longer, or that the performance will suck against the new official parser in development, or that nobody believes it. Or it really doesn’t matter.

But I say that when a language gets mainstream commercial support from one of the largest tech companies on the planet, that language has arrived and we should be grateful for it. I only hope the Java kids don’t steal our lunch money when we turn up and we all play nicely together.

The above link is to a thread that points eventually here and here that both help convince me this isn’t some warped joke.

I always thought there was something interesting about Sun. I think they might be about to become my favourite company again. I might even take another look at Solaris in coming weeks to see if they’ve made it suck less than the last time I played about five years ago. See – this is smart thinking on their part. They’ve got me interested in what they’re doing again. Tres cool, non?