Innovation in Software

No matter where it happens, if somebody out there is coming up with crazy ideas in software, I get to hear about it eventually. It’s just rare I get time to write it up.

This evening I caught on to Breach Candy Group’s idea for what they call AML or ‘Aesthetics Markup Language’ and decided to talk about it pretty much straight away. This article is under the “Philosophy” category for a reason. If you’re not in for some deep thinking, move along.

Their idea is to be able to define the aesthetics of a piece of film or art in a standardised markup language so that systems may be able to perhaps generate new content of a similar aesthetic style. For example, as they say:

“Let’s start with images. We could start off with the following variables:

• movement (speed of movement = speed of change in pixels?) This could be later used to analyze some rhythm of change.

• brightness and contrast (how would this be tracked = the relationship or average of pixels in any given location on the video?)

This could also later be use to analyze things such as harmony of composition, direction of lines in the mise en scene, etc. We would have to come up with a set of principles from art history and composition and see how these could be determined in the screen etc?

• color range (this would probably have to be RGB values in the image itself). This would probably move us into the realm of things such as monochromatic color schemes, bright colors, harmonious colors, contrasting / oppositional color … ie to use some notion of color theory to provide patterns in certain styles of video etc. I’ve studied this in high school so will be fun to revisit some principles of classical painting.

So I suspect what we need to do is set up a very simple experiment / structure in place that can be developed and extended depending on need. In other words, we need to develop … AML (Aesthetics Meta Language) … a basic language structure that would describe what the variables are within any analyzed video. This language, I suspect, could be then developed into the interface between the language of aesthetics and the computer.

Something like this:

//AML: “DEBBIE DOES DALLAS”

134
12
58
22

It’s a nice idea, but they’re missing a trick. The thinking that got them to this point to me seems much more interesting.

They argue that most of the complexity we witness in the World is a repetition of simple things that go on to form complexity. This theory exists behind cellular automata, fractals and more.

Cellular automata are incredibly important in our understanding of how complexity is created out of the simplest building blocks within the Universe. Take for example the image to the left of this text. It might look like something taken from a microscope, but it’s an image created in software using a CA routine that was less than 100 lines of code, simulating “dictyostelium slime mold”. You can find out more about exactly how it was constructed at the site I got the image from.

This idea of complexity emerging from simple rules is particularly important in the field of Artificial Intelligence for reasons described in a philosophically entertaining manner in the AML article:

“Here the key question is that what algorithms could be used to model the way humans think and thus be used to guide machines to perform complex tasks. The philosophical implications of this are even more profound than getting a robot to recognize faces or clean a non-linear toilet bowl. That is, if human intelligence is, in fact, highly programmable, what then defines humans from machines? This goes two ways: machines-as-humans and humans-as-machines. In other words, AI defines rationality a certain way with certain presupposition of what logic, thinking and consciousness are and how they can be pragmatically simulated in computers. But as importantly, if we look at the concept of rationality and how it has been historically constructed, this has always presupposes a certain “image of thought” that has excluded all that would not fit into the sphere of rationality (intuitions, insanity, madness, illogic, spontaneity, absurdity ….). So how would we then understand the blurred boundaries of man and computer (as intelligent forms, which neither technically speaking are) and the human-computer assemblage that is making the old notions of rationality/humanity perhaps increasingly difficult to defend? Humans as (programmed?) repetitions: computers as programmed repetitions: natural intelligence: artificial intelligence: natural stupidity: artificial stupidity …”

Think about that for a moment. The point is not to be “correct” in the sense of making an algorithm “smart”, but make it mimic so it has the ability to be just as insane, dumb and mad as humanity. If you understand the Turing Test correctly, we will have produced an algorithm capable of passing it correctly, when we produce an AI capable of insanity, melancholy and psychopathic behaviours. Asimov’s 3 rules of robotics can not exist in a machine capable of passing a Turing Test, in other words.

This is a good development of an argument on the part of the Breach Candy Group, partly because it’s intuitively correct but also because it follows the science we have to hand quite nicely. It’s a shame then that in itself it probably undermines the need for AML on which they built on these foundations.

AML is about how a computer would process very specific values – contrast ratios, line measurements, etc. – yet they accept themselves that what they need to do is instead mimic the way a human would describe the aesthetic. No film director would talk about precise values of contrast settings or line movement, but instead would find a way to talk about colour, tone, depth, warmth, speed and so on in a much more abstract way.

In other words, to achieve what they’re hoping for they need to develop an algorithm which is able to mimic the human way of parsing film aesthetics, train it by making it “watch” films and then ask it to produce something “like” a subset of them. They’re trying to find a way through by producing a way of retaining knowledge about an aesthetic in a standard form, but as anybody who has read “Zen and the Art of Motorcycle Maintenance” can tell you, there is much more to an aesthetic quality than how you describe it mechanically.

In fact, the point about a distinctive aesthetic quality is that whilst we know it when we see it, we all see something different and would all describe it differently. Let me try and make my point by using an image that has probably had its aesthetic qualities dissected, written about and analysed more than any other in the history of aesthetics.

The Wikipedia article on the painting has quite a detailed summary of some of the aesthetic judgements made. There are two groups these arguments can be placed: those that are algorithmic and those that are subjective.

It might be reasonable to produce in AML those that are algorithmic. They can be measured precisely – golden ratios, pyramid composition. However how exactly do you describe something as subjective as “the composition of the figure evokes an ambiguous effect: we are attracted to this mysterious woman but have to stay at a distance as if she were a divine creature” in a markup language?

I don’t have answers, only questions on this one. Interesting thoughts though. And if they can be resolved, we’d be a major step forward to understanding AI – and ourselves – much better.

Everybody loves video on the Internet – sites such as YouTube have shown the power of video, particularly in a network of people who are fed up with traditional media. However, it’s bandwidth-hungry, and low-quality webcams are the norm still. This means watching video on your PC is slow if it’s high-quality – a DVD quality BitTorrent can take days to download – and makes you feel sick if it’s fast but low-quality.

MotionDSP have created a new technology that tries to improve low-quality video by working out information that is ‘lost’ through aliasing of pixels. It’s impressive stuff, originally developed for the military. Now it is likely to find a home on the numerous video sites that are changing the way we all think think about film and video.

The demos look pretty good, but you can still tell there is something low quality about them – the blurred nature of the refined video makes me feel like I’ve lost my reading glasses and after a while, I think I’d feel a bit ill.