Innovation in Software

For a couple of years now, I’ve been fascinated by the possibilities of a field known as Augmented Reality. In a nutshell, AR allows a digital device to “overlay” digital information onto the physical world. This is quite cool stuff. Watch:

So, that’s nice and everything. We can take a digital device, and through the multitude of sensory input, we can start to use it as a viewfinder. We can even start coming up with zany ways of manipulating the image we’re seeing, which people with very little expertise are starting to play with:

What’s intrigued me though, is can we find more interesting ways to interact with the device. It’s great that I can point my iPhone at a piece of paper with a special block printed on it, and a wind farm comes out, or I can point it at the environment I’m stood in and get extra information that isn’t otherwise easily found. What else can I do? Well, thanks to the same technologies developed for those applications I can suddenly create a virtual reality headset (either to augment my environment or to travel to another distant one):

Again, pretty and interesting and opens all sorts of possibilities. But how much further can we go? I have no doubt we’re just seeing the early adopter stuff here, and that with time lots more interesting applications will become available utilising these technologies. Watch that space carefully.

One of the issues raised by this technology though, is our addiction to it and the fact we become chained to the device. All of us have suffered that weird syndrome that happens when out with friends and suddenly all of you in a group are staring at small boxes of plastic and metal, interacting with virtual worlds rather than the one you’re physically in. If we start to augment reality through this technology, do we lose something about the sense of place and interaction with the real physical World? Do we start to become machines ourselves?

Thankfully, a rather interesting prospect is on the horizon. Even better, it’s open source and achievable with cheapish hardware. It might take a moment for this to “click” with you, but this will likely be the most valuable 10 minutes you’ll spend this year on understanding the future of human interfaces with technology in the next few years. Trust me. If you haven’t clicked that link, do so before going on, or if you want to skip the background and just see it in action, here’s a demo clip with cheesy music for you:

The idea that the device disappears is not all that new – we have seen devices getting smaller and denser for years with that goal in mind – but the way this has been done fascinates me. This technology once developed a little more into something more consumable eradicates the need for a high-end smartphone, multi-touch technology like the $40,000 Microsoft Surface, portable media players, the lot.

What you need: a camera, a projector, a data connection and a pair of headphones, all plugged into something that can understand all of them. Right now, the hardware looks cumbersome, but that’s just a hardware engineering problem: there are already smaller projection devices and cameras coming to market that will make this technology nearly invisible when worn.

What you can do with it: anything a camera, mp3 player, iPhone, desktop PC, laptop, mobile phone, projector, surface table, AR application, in fact anything you can do with any digital device, all in one go. And all of it with the device being near invisible.

What you use it with: ordinary pieces of paper, walls, tables, your hands, the objects and people around you. Instead of staring at pieces of plastic, suddenly you are encouraged to look up at the World.

When something interesting happens you don’t fumble around, open a shutter, focus, click, crop, tag and post. You just make a shape with your hands, and there’s your photo. When you want a flight-time update you don’t fumble, open an app, type, click, read: you just look at your boarding pass. You’re in a bookshop and you wonder if the book in your hands is any good so you fumble, type (or perhaps photo the barcode), click and read, perhaps clicking around a few stores on a small screen. With this, you just hold the book. That’s it.

You might think this is sci-fi, that nobody would ever use all of this or that the “back-end” needed would be too bulky. I would disagree. I would say it’s one of the most interesting developments in technology this decade. I will be watching for the release of the source code due soon with some interest. Pranav’s site is a good place to keep your ear to the ground.

Many years ago I ran FreeBSD on pretty much every system I touched. I loved it. The lack of political polarisation that touches the GNU/Linux community combined with the power of Unix under the hood sucked me in.

I even, almost infamously, had a rant about another operating system I now use daily. I was a zealot. I am ashamed of my attitude in that email – my only excuse is that I was still quite young, a rather naive and opinionated 26 year old.

Today, I use exclusively OS X, and occasionally boot up a virtual Windows instance to test code on MS code. I have a terminal open nearly all the time and often dive down to the Unix goodness, but more often than not the bulk of my work is in the browser, text editor, IDE or Mail app.

Why then, I was asked the other day, do I not just ditch Apple and get a Linux (or even BSD), machine to work with? The bulk of my work (except for iPhone application development) would continue as before.

My answer is very simple: carpenters don’t build lathes.

To explain, let’s first go back to that rant I had about OS X. About a month later, I ordered my first Apple laptop – an iBook G4 – and was very happy with it. What happened in that month?

I realised when shopping around, I needed to check out the chipsets used for WiFi in each laptop because FreeBSD wireless support was pretty hit-and-miss back then. I also needed to understand the graphics chip-set because I’d need to compile X if I wanted to run a graphical desktop environment. I then realised I’d need to assess pretty much every feature in terms of hardware compatibility to make a purchasing decision. And then, on receipt of my new laptop I would have to spend a few hours doing all the compiling and fixing, and I would then need to do this work every few months as part of an upgrade cycle.

Yes, I’d get the power and flexibility of my own tailored operating system environment, but isn’t that a lot of work?

I recalled a few months previously a friend was playing with his new smart phone. It was running Symbian and he seemed to be doing lots of prodding and poking with it. Enquiring what he was doing, he said he was “doing some maintenance” to keep it in working order. Hang on, was he effectively sysadmin’ing his phone? “Yes, I suppose”, came the reply. When you have to do systems administration work on your phone, your phone is no longer a tool to assist with your work, it is an object of work in itself.

It was this insight when thinking about my operating system choices that directed me to OS X: I wanted the power of Unix, sure, but I wanted it to just work. I wanted to be able to get on with my work, the laptop and operating system as tools rather than objects of work themselves.

This small insight has made my life a great deal easier, and still dictates not using any other operating system. Some of my peers see it as weakness (even more point to my original rant above), but I see it as spending my time doing the things I love.

And it’s also an insight I think that as developers we forget: are we developing tools that assist with objectives, or tools that are objects of work themselves. Are we building interfaces and suggesting business logic that means our customers spend time managing the behaviour to fit around them? If so, why?

In the last few weeks I’ve seen some really interesting “reductions” in functionality that aim to make tools more directly useful, rather than requiring some administration.

Take for example, the role of authentication. You need to “contain” all of your customer’s “stuff” in a way that is linked to their “account”. So we have user registration, user profiles, account activations, password resets, etc. Seems like a lot of stuff. Are we sure we need it?

Take for example, Posterous. Their home page explains it all:

All you need to do is email stuff to post@posterous.com and you’re up and running. No signup, no captchas, no password strength indicators unless you want to add them. You’re a person capturing stuff – why do you need to admin yet another web application?

BootStrap UK (which is worthy of an article in its own right as a concept), needs accounts but has a simpler registration system: just follow @bootstrapuk on Twitter and they’ll follow you and direct message you a password. Done.

There are other examples out there as well, sometimes using software to help break out “lathe obsession” elsewhere in society. One iPhone app I’ve seen reviewed gets the banking system to work around the customer, rather than demand the customer head into a branch to suit the banking system. Brilliant.

We often spend too much time trying to work out how a customers will need to behave around our application, rather than how we need to get out of the way and let our customers use what we produce in order to do their thing. In every customer we need to start seeing the carpenter more, and assume they – perhaps unlike many of us developers – don’t like tinkering with lathes for its own sake.

One of the advantages of being almost 30 is that people less than a decade younger than you tend to think of you as being “wise”. Some of the staff in my local bar will ask me about everything from US politics, the Renaissance, Alan Turing, 1980s TV commercials and arcane facts about the early forms of Parliament. Cultured bunch, the staff in my local.

Last night however, it was my turn to learn. One of them had asked me last week about the phrase “Jumping the Shark” and where it had come from. Last night she told me the very next day after I’d explained it to her, she watched an episode of Scooby Doo (OK, maybe they’re not that cultured) where Scooby jumps a shark and that it had made more sense to her knowing what it was a reference to – it is one of the classic insider jokes within TV comedy. I then had to re-explain it all to the other people assembled. The conversation that followed was… interesting:

It went on in a similar vein for a few more minutes. More examples of the futility of the network, the silliness of the apps. Admittedly, none of them had left Facebook yet, but that might be that it’s rather hard to leave, as GeekUp and Co-working day regular Alan Burlison found out

These are people the social networks need. In their early 20s. University students. Bright, intelligent, aspirational. I have no doubt that within a decade most of them will be in the upper 25% of earners in the UK. Malcolm Gladwell would call them “sneezers” or something – they spread their likes and dislikes around their friends quickly. They set trends.

And in the last couple of months they have come to hate Facebook and MySpace.

Specifically, they hate that these networks have been opened up to people engaging in what is effectively a developed and sophisticated form of spam. They hate that they are being hassled via the social graph into doing “fun” things that are actually about as fun as receiving a hoax virus e-mail. They understand that their time and attention is important and its being wasted by sites that don’t respect that.

I have ideas for applications that will actually add value to the social graph and be of use to people in this group, but by the time I get to roll them out it could be too late – the people that make the platform interesting to me as a recruitment base for customers may have moved onto something else.

Facebook are adding features to improve the user experience as they learn how developers are gaming the system. They might win the battle in time, but ultimately they might have to give more control to users to block invites from apps that are not even remotely in their realm of interest.

This isn’t over yet. 2008 could easily be the year the social networks died.

This afternoon, I’ve been playing around with Facebook’s ad platform. Partly for Vagueware, partly for other businesses, I’ve been looking at what Facebook says about its user base to advertisers.

The level of targeting is just outright astonishing. It allows for ads not only to be targeted on demographics such as age range and city, but even on interests and relationship status.

For example, I now know there are approximately (all figures given are approximate to the nearest 20 or so), 120 people in Manchester interested in Programming.

Out of the 2,017,440 UK citizens who describe themselves on Facebook as ‘single’, 998,900 are male, 904,960 are female. The numbers don’t add up because some people don’t define a gender which makes the point that if you don’t fill info in, you can’t be targeted via that info.

There are 1,180 females in the UK who declare an interest in ‘Computers’. The figure for males is around 8,540.

580 UK men say they’re really into shoes, with 14,300 British women aspiring to be Imelda Marcos.

There are around 5,680 people working for BT in the UK on Facebook. In the US, there are around 40 people working for O’Reilly Media who mention it in their profile. I could target either with an advert – handy if you have a product or idea you want to pitch.

This is just the tip of the iceberg. Advertisers don’t need to know who your friends are (the social graph), to target you this tightly. If a member of GeekUp wanted to put up a singles ad for all single women between the ages of 24 and 32 who are into computers resident in Manchester (approx. 100 of them), they now theoretically could. Lucky ladies.

The question is, is this really a bad thing? Doesn’t it mean we’re not all going to see advertising that really has no relevance to us? Or does this kind of marketing mean that we are the perfect willing victims for advertisers to go deep into our psyche? I knew this day was coming, but I thought it was still some way off.

Vagueware is not my only gig. My other Directorship is very low-key, doesn’t take much time, and is where I and my business partner experiment with various marketing revenue models. It’s primarily been a learning experience, and the frustration we’ve had over the last two years in getting various complex projects rolled out has meant we’ve been looking at partnering with technology companies and focusing on the marketing and customer communications side.

Our latest venture is quite a departure for us. Excuse me whilst I shill for a couple of paragraphs:

Whilst cash back websites are not brand new, we’re hoping that with a really solid technology platform underneath us, we’re going to be able to do something special in the way of helping people make shopping a little more fun – and save cash too. I do however, need some eyes and ears because I’ve had no control over technology roll-out, so I’m interested to hear of problems people might have.

As you can see, the concept is really simple. You sign up, we give you a fiver. You shop online, we give you a percentage of what you spend. You refer friends, we give them a fiver, and we give you a fiver as well once they qualify for payout. We’ll point you in the direction of freebies that pay you money as well. We’ll be launching a blog to highlight particularly good offers. Occasionally e-mails with super secret codes will land in your inbox and you will consider yourself a wise old bean for signing up with us. It makes things cheaper if you’re doing a lot of Christmas shopping online, although for various reasons we’re late to the party for that one, so our strategy is a little more long-term.

End of shill

I mention it here, because I’m interested in problems an educated audience (that’s you, dear reader), might see. We know for example that the back end systems are rock-solid and everything is nice and secure, but are there ‘quirks’ we’ve yet to spot that only a geek can spot? Maybe you just think the business model is odd, or we haven’t explained it very well. Either way, I wanted people whose opinions I respect to take a look before the big marketing push over the next 12 months, and see where we can make improvements.

A friend just forwarded to me an article called “The Outsourced Brain” over at the New York Times. A sample:

” Until that moment, I had thought that the magic of the information age was that it allowed us to know more, but then I realized the magic of the information age is that it allows us to know less. It provides us with external cognitive servants — silicon memory systems, collaborative online filters, consumer preference algorithms and networked knowledge. We can burden these servants and liberate ourselves.

Musical taste? I have externalized it. Now I just log on to iTunes and it tells me what I like.”

This is going to gradually become a debate over the next few years as we pass more and more of our thinking and life over to algorithms. Stroustrup once said “Software runs civilisation”. I think we’re approaching the point where we can say “Software runs civilians”. There are obviously issues with this that need to be explored.

About a year ago I developed a hypothesis of what humanity would broadly look like 100 years from now. Some friends found my synopsis of this vision a little ridiculous: “You know the borg in Star Trek? That’ll be us”.

What I mean is that we are slowly moving our thinking out into the cloud and acting as one. Individualism is being lost, group-think is being encouraged. If that sounds a little Orwellian, can I just point out that we’re the ones encouraging it on ourselves – from CCTV cameras to collaborative filtering on Amazon – it is not being imposed on us.

The irony is that for all the menace of Borg assimilation and Orwellian dystopia in fiction, we are shaping parts of our society into something that mimics it in the hope it will lead to peace and harmony within society. Maybe it will, I don’t know.

The hope we have is that if we spend less time thinking about what music to buy, which directions to use to get somewhere, and trying to remember things we can get out of Google and Wikipedia anyway, we’ll have time for more important things. The question is what things are we doing with that time? Are we just filling that brain capacity with other trivia we don’t need?

Not for the first time, I feel that those of us styled “Software Engineers” have a responsibility to ask some questions here.

Let’s take a couple of ideas driving Innovation in the software arena right now:

• Work should be more like play
• 3D alternate Worlds are useful in some way
• People are finding it difficult to deal with the incoming flow of information

Each on their own can lead to ideas like Amazon’s Mechanical Turk, Second Life or better Bayesian filtering. In short, when you focus on an idea you can find ways of making software better. Some people ask themselves, wouldn’t it be great if you mixed some of them up? Say a 3D World where you work? Or handling information flows like a game?

What happens when you try and mix all three up? Well, I concur with TechCrunch when I say this example is just pure evil.

Yes, I can’t quite believe it myself. And I thought the ads in GMail would harm productivity in a mail application…

Tim O’Reilly is calling for Facebook to share social graph data so that systems can leverage all the data you’ve shoved into Facebook and use it within their own apps.

There are a couple of issues here.

Firstly, Facebook isn’t actually stopping 3rd-party API developers from knowing who your friends are, and if your friends agree to add an application, the app provider can see their graph too. What isn’t agreed yet is whether this should be made more open, or whether there needs to be a standard way of describing this data. There are all sorts of reasons why I might not want my “social graph” to be made available in an easily-manageable format, not least because it raises privacy concerns.

There is also the fact that Facebook’s business model relies on not making this data available. The “expose your data, and they will come” argument relies on a simple metric of conversion.

Within a company like Amazon, exposing the product catalogue by API is a no-brainer. The more places their stock list is available, the more chances they have of getting somebody into the system, the more likely they are to convert them into a sale. The porous membrane an API gives an app developer in this instance means 3rd-prty developers do the hard work of getting stock shifted in countless innovative ways the original company wouldn’t have thought of.

Facebook however, is different. The ‘conversion’ in their instance is getting somebody to look at pages with adverts on it. What they need is for their users to actively recruit more users – invite them inside the walled garden – and then try and keep them there. They’ve out-sourced the “retaining” part of the equation to developers (playing games, taking quizzes, sharing links, glorified e-mail), but by allowing their most valuable asset to be easily exported they are reducing their customer’s incentive to stay within the walled garden.

As always, it comes down to whether you have a right to that data, and whether you have a right to move it. I’d argue you do, but I’m suggesting it’s going to be hard for Facebook to allow you to take it wherever you want.

[UPDATE]: I realised there is a way to do this without Facebook’s permission. I’ve written it up on the site.

Apple Insider is giving us a thorough run-down on the history of the Virtual Desktop in anticipation of Apple’s launch of OS X Leopard. Leopard will ship with ‘Spaces’, Apple’s own take on the Virtual Desktop.

What has surprised me most in recent years is that nobody seems interested in why we continue to try and fit everything into the ‘Desktop’ metaphor at all.

As I understand it, when they were trying to decide an interface at Xerox PARC, they thought about who would use the tools they were developing. It is a sign of the times that they thought the only person likely to be sat in front of a keyboard any time in the future would be a secretary – so the developers went down to the typing pool and looked at how a typist might understand the World of work.

It’s for that reason, and that reason alone, we have the metaphor of a ‘file’ and a ‘folder’ and a ‘desktop’ within modern user interfaces.

There have been experimental interfaces to shift the metaphor, but these experiments have normally failed because the chosen metaphor has been so weak. Surely there has to be a better way to think about the way we work now we have fully embraced the hypertext document, the network as the computer and the social network?

A few years from now, people new to the Internet will find it impossible to imagine an era before video online. What use is that content though, if your user is visually impaired? How do we go about indexing and searching it?

A few weeks ago IBM announced an “Accessibility Internet Browser for Multimedia” over at alphaWorks. It addresses the short-comings of JAWS or voice-enabled browsers, and is built on top to of the Eclipse Rich Client Platform as a stand-alone application. It means the play/pause buttons (normally unavailable to visually impaired users) are stripped out and made accessible, and that playback doesn’t ‘clash’ with voice-synthesised browser operation.

Developers can add metadata to reorganise or simplify the content, provide additional information, add special navigation or even provide audio descriptions for movies using XML metadata.

It’s this last point that I think is going to be interesting. If this tool – or one like it – becomes standard, and video producers are encouraged to mark-up their content properly, existing search and relationship algorithms can be applied to video content. Right now searching video content is pretty limited – maybe by making it accessible, we all benefit.