Innovation in Software

For various reasons I’ve decided to change the comments system on this blog to use Disqus. That means there is much more social media cohesion within the comments, and hopefully we’re going to be able to have a better 2-way discussion here.

Let’s get started: It seems twitter traffic might be peaking right now. So, I posit that all people who have access to the Internet today who want a twitter account (or indeed any other social media account), has one. There are few new users who are likely to sign up. Therefore the only growth available for such sites, is new Internet markets where access is limited (e.g. Africa). Discuss.

For some reason, Mephisto isn’t behaving on my server right now, so comments aren’t working. I plan to move the whole shebang over to Wordpress sometime “soon”. That said, I’d like as a gentle introduction back into me blogging here again about “Innovation in Software” (less ‘what is happening in Manchester’ in future), to talk about comment systems.

This cartoon from the excellent xkcd strip sums up my problem with comments right now:

The problem I have is this: on any popular system where users are allowed to comment, as the number of users able to comment without fear of genuine peer review increases, the signal-to-noise ratio drops exponentially.

In other words, if I and 5 million other people can comment on a YouTube video without any fear of us being reminded of what we said not just in the future but the very next time we meet a friend, we are more likely to be flippant, irrelevant, and “noisy” than if we knew people whose opinion of ourselves mattered to us were going to be reading that comment and evaluating it.

It’s why social network status updates and posted items are relatively sane and measured and why blog posts are more considered. We care about what the readers think, because what they think will have a direct impact on our future relationship with them.

So, whilst thinking about kagtum a lot recently (background if you’re unaware), I’ve been thinking about this problem. How do you allow for user comments without them descending into noise?

The “mission statement” for kagtum in its current form is something along the lines of “delivering relevant news and event information”, where “relevancy” is the secret sauce that gets quite complicated. How do we make sure every comment you see is relevant in order for it to stay within that mission statement?

I have a possible answer, but I need to keep it close to my chest for now. Normally my ideas are thrown out into the wind as being worthless, however my answer has a direct consequence on execution of a business plan. That said, if you come and meet me at an event and ask me, I’ll tell you what it is if I trust you. ;-)

[UPDATE]: within 12 hours of fixing it, I’m getting the spam bots floating by. Yay. It might be time for a move to something that support Akismet…

When I turned comments back on I noticed that leaving more than one comment wouldn’t work. Apparently this is a known issue in Mephisto under certain conditions (I was at the time using Apache22 + fcgi), but nobody is completely sure how to fix it. It was time to move over to Mongrel anyway, so now that move has happened, everything is fine.

The last week, incidentally, has been hellish. I’ve found it hard to get the time needed to do all sorts of things, including the 1st October launch of the main site, but I now have a relatively quiet weekend, in that I’m only going to be working 10-15 hours over the course of it. That means I can make some progress on other areas. Yes, that includes deploying this code. Stay tuned, but with all this expectation you might by now be expecting something polished – it really is alpha.

To a weekend of success then…

I turned off blog comments a little over two months ago [and explained my reasons why] at the time. Virtually everybody I personally know who reads this blog disagrees with that decision. Most of them never commented, but that’s not the point. They feel there is an injustice in the fact they can’t unless they blog about it themselves.

My life has been a little easier without having to log in daily and moderate the spam out of the system, but overall I think it’s harming the blog not to let people give some feedback directly. I’d love it if they took the time to write up an article on their own blog and link back, but that’s just not happening. It’s time for a re-group.

From this article forward, comments are back, and will never close on any article.

They will be moderated, and I’m going to be inclined to block one-liners or unrelated comments, whether they be spam or not. Of course, even if your comment doesn’t get published I’ll still read it, so feel free to be abusive if you wish. :-)

[UPDATE]: it would appear the comments stuff got broke whilst having a rest. No more than one comment gets attached to each post for some reason. I’ll look into it ASAP. So, for now, first mover wins. :-)

As a rule, I hate blog posts about blogging. I also particularly hate blogs that cite posts that in themselves are citations on other blog posts. It is then, with regretful hypocrisy that I find myself citing Joel Spolsky whose post in turn cites Dave Winer. Even worse, Joel is citing a blog post made nearly 8 months ago. That said, I think the point being made is of interest and might change the way we think about user-generated content over the next 12 months.

Joel’s main point is that comments are next to pointless. He doesn’t allow comments on his blog, neither does Dave Winer. Their argument is that as a means to self-expression, the author has a right not to be bombarded with people shouting them down and they shouldn’t have to see “their space” polluted with garbage.

The flaw with this idea is the idea that blogs are a means of self-expression. They are not. They are a way to find like-minded people and to provide a low barrier of entry to getting involved in w global discussion about the things you care about. They are not billboards, they are not magazines, they are not leader columns in newspapers. Nor are they journals or fascinating insights into your unique and tortured soul (you delicate little snowflake, you).

They are about a two-way dialogue, a means to advance ideas and to further understanding in the most cost-effective and uncensored way possible. I could produce a magazine about innovation in software but that would bankrupt me – instead, when I get time, I put articles up here as and when. I might not get tens of thousands of readers, but I certainly get hundreds (in fact, 2,167 unique visitors in the last 30 days have visited blog.vagueware.com which is not too shabby). And some of them – very occasionally – leave a comment here.

The purpose of all this is to encourage a conversation. Comments and trackbacks are typically an important part of inviting people and saying “Please! Come on in and tell me what you think!” and help move things along.

The real problem is that really interesting ideas are never in the comments. I don’t get the turf wars that spark off on other blogs because every comment is moderated – I don’t allow through spam, abuse, or one-liners and so the comments are worth reading normally, but normally nobody ever reads them. The comments are typically idle thoughts, occasional insights, sometimes intelligent but often just a throwaway quip. They intrigue me, but the comments are not what excites me about the conversation around here – it’s the people who link here.

When somebody sits down and decides to push the conversation further by spending 30 minutes writing a piece about something I’ve written, edited it, made a decision that they’d be happy for their name to be next to those words for all eternity (the Web never forgets), and then publishes it with a link back to here, the quality is generally higher than somebody who quickly fills in a little box at the bottom of the page.

I haven’t had many inbound links, but the ones I’ve had have always made me think more about what I do and why I do it than any comment ever has. There used to be a load of links when this blog lived at a different URL, fewer now, but in time I expect that’s where the interesting ideas will come from.

This line of thinking seems to be an emerging trend out there.

Recently, there has been a lot of love for Tumblr, a blogging platform without comment facilities. I think it’s a daring tactic, and one that will improve the quality of the content they carry. It’s also wise engineering with a direct impact on profitability. All of a sudden they don’t need to worry (quite as much) about spam. They don’t need to worry (quite as much) about authentication. They need less storage on their servers. The code is simpler. It even helps their authors. If I see something on a tumblr blog I have a choice:

• Do nothing
• Link to it in an article here that expands my own thoughts on the subject

Either way, the noise level within the conversation is minimal and providing I’m not an illiterate moron, the signal goes up. What’s more, the author of the tumblr blog doesn’t just get his own audience thinking about what he wrote, but perhaps some of mine as well. A few people might link to my piece so I get their audience, and so does the tumblr blog. This is how it’s meant to happen.

The author has no worries about spam, or about people cluttering up their space with thoughts that just don’t “get it”. They just get to go on linking to people and watching out for who links to them by way of Technorati or similar.

In short, comments aren’t a bad idea because you have a right to not be shouted down, they’re a bad idea because they slow the conversation down. They raise noise, deplete signal and keep the A-list static.

As such, I’ve decided that as of today comments are closed here. It makes my life easier, and hopefully yours too. If you’re a friend of mine on Facebook, you can make comments on the notes that get imported from here automatically if you want, but if you really have something to say about anything you read here, I’d ask you to post a link on your own blog. I’ll see it (I track every inbound link) and if I think it’s relevant I’ll update the article you point to. I may even write a new article pointing back at you. We both benefit.

If you don’t have a blog, I have to ask: what on earth are you waiting for? Tumblr awaits…

Some people reported problems leaving comments. I was upgrading Mephisto and so Rails was chucking up all over the logs at one point [note to self: check bang lines on dispatch.cgi next time] and as a result comments, RSS feeds and even the admin control panel I use to manage it all (eek!) were not working as planned. In fact, I’m amazed anything worked at all.

You will be pleased to note however that all posts now have comments working perfectly fine on them, just click on the link at the bottom. However, please bear in mind that anything non-constructive will be deleted – this isn’t a democracy, it’s my business.