Dr. Benedikt von Walter - head of MTV's digital research team for Northern Europe

So as expected: there has been quite a lot of the usual hooblah and blaahblaah about innovation and digital technology and other such buzzwords that everybody seems to spit out these days (what do these words even mean anymore?).  I guess when you have worked in this area as long as I have, the intellectual output of such workshops is seldom the main point any more. Rather, such workshops provide us with the best way to experience the different kinds of creative energies and experiments that are going on in regions such as the Baltic States.  So despite some of the usual recycling of American cliches of digital openness, participation and digital revolution plus a few good speakers, the best part of the week so far has been the guided visits to the local incubators, art collectives and cultural centres such as the Culture Factory Polymer where probably the most interesting art/technology/business synergies are happening today.

The first week in Estonia is now done. Now looking forward to seeing a few more of such places in the lovely town of Riga.

There are several companies and groups that are now focussing on this – one of the best examples of it being Pachube. There is no doubt that there are massive gains to opening up communication channels between various ‘active’ objects. Applications lie in several domains: energy, environment, SCM, reactive systems, so on and so forth.

Personally, I expect two primary variations of this:

Closed Systems: where organizations control their own silo of communication channels and devices. Example: yellow cab state and position monitored by a central monitoring system in the yellow cab company. Encrypted channels, and everything that comes with closed environments. This is already happening, btw.

Open Systems: this is the interesting bit – imagine being able to predict earthquakes (through resources like LISS, for instance) and tweeting it out. Or collision detection in traffic. Or reactive rooms, environments.

Post the meetup, I decided to chalk out the underlying architectural implications for  an open web of highly communicative systems, with the goal of opening up thinking around some issues that are bound to come about:

1. Unifying the communication language: we know about microformats and how long they have been in the pipeline. Especially for open systems that provide data feed, this is essential.
2. Handling data explosion that comes with Real Time systems – take the case of twitter, for instance. The scalability challenges they have faced are widely known. And this is when they aren’t doing any semantics yet.
3. Creating Relevance: as data explodes in volume, applications need to figure out a way to crunch and filter out the noise. There is a google right there in the making.
4. Smart usable applications: while you could create an application that makes your ‘car keys’ searchable by the public, you wouldn’t do that now, would you? Technology is not the biggest problem, creativity is.
5. Data presentation: interface is the king. With the volume of data, it becomes imperative that the interface / application is able to understand the deeper chaotic nuances of a ’state-based’ system, and make intelligent interfaces out of it.

One of the key idea that Kit Macgillivray of Real Time Project discussed was the concept of gazillion agents do one simple task, and yet, as a collaborative system, appears very smart.

I will be posting more on this later.

The idea is simple: I create a set of microprograms (I fondly call them ‘brats’!), each of which have their own selfish agendas, decision processes, survival rules, and their own I/O probes that constantly monitor the surroundings for resources they need (and perish soon if they don’t find them)… and then I spawn each variety of agent multiple times and see an overall behavior emerge.

The first set of such agents are already sweating their necks, making highly stupid decisions… and yet surviving, but my goal is to now take it a step further.

I now I want to go in two directions:

- train a GA or reinforcement learning based system to figure out the initial configurations.

- spread the agents across multiple computers on the web, thus creating a virtual cluster, an ecosystem that would be very interesting to study.

You may ask, why is this any different than Cellular Automata, the system that Stephen Wolfram brought back into fad through his book ‘New Kind of Science’? The fact is, though similarities may exist on the surface, a deeper inspection would reveal that my agents are far more inspired by ant colony / multi-agent based emergent behavior systems than CA — the only difference being, I am trying to far more flexible about the toolkit, instead of sticking with just pheromones!. Also, at the end of the day, I am just trying to learn my tools!

[Tools used: Ruby, RubyGems, MySQL, a lot of caffeine, and box dvd set of The Wire]

If you love to use Orkut, Facebook, Myspace, Twitter, Flickr etc. and would love to create similarly exciting social networking and Web 2.0 products for the web then here is your chance to be a part of action. Google.com came out of an initiative by two students to create a university wide search. We believe in such initiatives, where there is a compulsion to use the web for something useful.

The deal you get:

We get:

What are we looking for :-

Who should apply :-

From essentially zero, we’ve passed a watershed of more than 3.3 billion active cellphones on a planet of some 6.6 billion humans in about 26 years. This is the fastest global diffusion of any technology in human history — faster even than the polio vaccine.

“We knew this was going to happen a few years ago. And we know how it will end,” says Eric Schmidt, chairman of the board and chief executive officer of Google. “It will end with 5 billion out of the 6″ with cellphones. “A reasonable prediction is 4 billion in the next few years — the current proposal is 4 billion by 2010. And then the final billion or so within a few years thereafter.
(LINK)

Wherever you look and read, it seems mobiles are emerging as one of the key technologies that we need to reckon with. But when my friends ask me, what do you really do? I tell them that – when in my research mode – I am interested in cutting-edge technology, future media and emerging digital cultures in the broad sense of the term. And then I pull out my own mobile – an old Nokia that has no additional functions excect shock protection and the ever-so-important flashlight. So I suppose I need to soon stop being the quentessential abstract academic who only talks about things and actually now get me one of these multi-function sleek sexy phones that I always predict that will probably have the most significant impact on how people communicate in the future. Just to have a look what the future feels like.

The first is that “things” (in technical language “identities” or “forms”) are the emergent outcomes of complex repetition of patterns through which complexity if formed. These can then be reverse engineered and computationally modelled so that, say, a fractal algorithm can be used to model complex patterns in nature. This is especially common today in some of the 3D and visual compositing software that we use today and all kinds of experiments with genetic algorithms and cellular automata etc – simple rules, complex outcomes.

The second is that human intelligence itself is made of, to a large degree, of such repetitions. This idea is rather much more elusive than the first but is one of the key presuppositions behind artificial intelligence research. Here the key question is that what algorithms could be used to model the way humans think and thus be used to guide machines to perform complex tasks. The philosophical implications of this are even more profound than getting a robot to recognize faces or clean a non-linear toilet bowl. That is, if human intelligence is, in fact, highly programmable, what then defines humans from machines? This goes two ways: machines-as-humans and humans-as-machines. In other words, AI defines rationality a certain way with certain presupposition of what logic, thinking and consciousness are and how they can be pragmatically simulated in computers. But as importantly, if we look at the concept of rationality and how it has been historically constructed, this has always presupposes a certain “image of thought” that has excluded all that would not fit into the sphere of rationality (intuitions, insanity, madness, illogic, spontaneity, absurdity ….). So how would we then understand the blurred boundaries of man and computer (as intelligent forms, which neither technically speaking are) and the human-computer assemblage that is making the old notions of rationality/humanity perhaps increasingly difficult to defend? Humans as (programmed?) repetitions: computers as programmed repetitions: natural intelligence: artificial intelligence: natural stupidity: artificial stupidity …

Anyway, all this probably seems rather abstract here. However, what was a concrete outcome of this philosophical babble was a project called AML (or what we like to tentatively call Aesthetics Mark-Up Language) that we are now currently trying to get our head around and develop. Similar to the more philosophical questions above, the idea here would be to experiment with how we detect patterns in certain visual styles such as certain genres in film etc. These patters could be then transposed/translated to other instances so that, say – yes, however preposterous this may sound – a Bollywood Film could edit a Hollywood film!

To perhaps see more what we mean by this, please find below an excerpt of some of the behind-the-scenes work-in-progress. The email conversation explains the idea better than I could re-write.

In film theory, for instance, the following elements are often talked about in the semiotics of the filmic image (well it is more complex but bear with me for the time being …)

1) Images;
2) Phonetics = speech;
3) Noise = background noise etc;
4) Text
5) Music.

For our purposes, we will probably have to stick to images for now. Tracking classical elements of film, in the beginning, is far too complex to get started with. Such as depth of field, montage / contrast etc. So what could be easily tracked to get started?

Let’s start with images. We could start off with the following variables:

- movement (speed of movement = speed of change in pixels?) This could be later used to analyze some rhythm of change.

- brightness and contrast (how would this be tracked = the relationship or average of pixels in any given location on the video?)

This could also later be use to analyze things such as harmony of composition, direction of lines in the mise en scene, etc. We would have to come up with a set of principles from art history and composition and see how these could be determined in the screen etc?

- color range (this would probably have to be RGB values in the image itself). This would probably move us into the realm of things such as monochromatic color schemes, bright colors, harmonious colors, contrasting / oppositional color … ie to use some notion of color theory to provide patterns in certain styles of video etc. I’ve
studied this in high school so will be fun to revisit some principles of classical painting.

So I suspect what we need to do is set up a very simple experiment / structure in place that can be developed and extended depending on need. In other words, we need to develop … AML (Aesthetics Meta Language) … a basic language structure that would describe what the variables are within any analyzed video. This language, I suspect, could be then developed into the interface between the language of aesthetics and the computer. Something like this:

//AML: “DEBBIE DOES DALLAS”

<contrast>
<high>134</high>
<low>12</low>
<average>58</average>
<mean>22</mean>
</contrast>

You get the point. The interesting thing about such an approach would be that we would be developing an entire syntactics of the aesthetics-computer interface that could be taken to any given direction we want. And then, of course, transposed back to how we would edit any clips or piece of media that we choose to run through the framework.

So how would this relate to the aesthetics machine? Basically, determining before hand the parameters for aesthetics is impossible. In film theory, there are some ideas such as structuralist semiotics that try to describe the language of different films but – as you know – aesthetics is notoriously difficult to pinpoint, especially when we are stuck with pixel-level pattern recognition on the computer. So what the aesthetics machine could be is the extension of the AML meta-language that would emerge from experimenting with as much visual material as possible. Say, we make AML public and get film-buffs and media-freaks to go through the entire ouevre of, say, German Porn genre to see what patterns emerge. These are then described in the wider structure of the AML language that we start developing So we get:

//AML: “DEBBIE DOES DALLAS” AESTHETIC STRUCTURE

<crossfade>
<number>12</number>
<duration>145</duration>
</crossface>

<contrast>
<high>134</high>
<low>12</low>
<average>58</average>
<mean>22</mean>
</contrast>

etc etc. Perhaps we could even open this language up so that some elements could be done analogically by volunteers (ie Amazon Turk Model) to further build the language beyond what is possible through pattern recognition. As this language develops, then the aesthetics machine will then merely be the instantiation and translation of these variables to other media material.

So if this approach would be the best, we need to develop / conceptualise three things:

1) The pattern recognition engine to get started experimentation with
simple data;

2) The syntactic structure that would describe any given visual material and that could be populated both by computers as well as analog humans;

3) The basic experimentations of how the AML language could be then
used to edit / re-mix some other media element.

So one reason why we are interested in Processing is exactly that it is the ideal platform for such experiments to emerge. In any case, watch the space here as our ideas eventually develop and we start getting the first concrete experiments up.

EMERGING DIGITAL CULTURES IN ASIA AND AFRICA – THEORY AND PRACTICE

DRAFT V 1.0
JANUARY 28, 2008

OVERVIEW:

The course Media 2.0 in the Global Context looks at the emergence of new media technologies from a distinctly global perspective. We have all heard the buzzwords recently: web 2.0, wikis, Second Life, social software, blogos, locative media, multiplayer online role playing games (MMORPG), mobile convergence etc. Much of the debates, however, has so far focused predominantly on the Euro-American context. Yet some of the most interesting developments are happening elsewhere. For instance, in Korea online multiplayer games are now more popular than television among young people. In sub-Saharan Africa, the explosion in the use of mobile phones has led to new innovative business models where paid airtime is now used as a new currency for cross-border interaction and banking. In India, software engineers are busy coding the future applications of the Internet of tomorrow. In Iran, weblogs have emerged as a new turbulent political space where future of the country is debated. What unites all these divergent media is the challenge they pose for media research in the 21st century. The course therefore asks: are the methods and theories that emerged out of the legacy of older media systems such as print, radio and television still applicable to understand the impact of some of these new developments globally? And if not, how should research adapt to the logic of what we could call broadly media 2.0; that is, the emergent new media technologies that increasingly mediate lives across the world?

With this in mind, this course aims at unraveling some of the methodological and theoretical problems that are raised when we look at what we could loosely call media 2.0 in the global context. The course, however, does not pretend to provide a final word on this shifting and emerging field. Unlike many other introductory courses to “digital culture” and/or “new media” (in many variations) the course instead aims at providing students with the necessary tools that allow them to begin investigating the issues that get raised for themselves. Its focus is therefore as much experiential and experimental as it theoretical and academic. It aims at providing both the theoretical background to some of the key debates dealing with contemporary digital cultures globally today but as importantly the practical know-how for students to begin using and exploring these new technologies first hand. It therefore encourages students actively involved in the debates that are taking place around new media technologies globally as well as use these new technologies for conducting, promoting and publishing research in multiple environments.

What fascinates me about this rapid escalation of mobile phones especially in the poorer regions of the planet is that it becomes rapidly appropriated to accomodate to the local conditions. The problem with many of our understandings of modern technology comes from the fact that we – as a part of the global information elite located in the cushier sides of the digital divide – have been accustomed to seeing the world through the prism of our broadband, blogs, Second Life, IM, email, Twitters, Jaikus, Internet-mobile convergence etc. Because of this, we have a hard time to imagine how, for instance, mobile phone use changes drastically when there is no steady electricity (in Africa, entrepreneurs have come up with car battery-operated charging stations on motorbikes) or proper infrastructure (cross-national trade now is being done in Kenya by sending paid airtime as a viable currency for payment).

Therefore, reading Jonathan Donner’s research into the use of mobile phones in sub-Saharan Africa is like pressing the refresh button on the browser. Donner argues that in Rwanda, “beeping” has emerged as a mainstream activity as a way of staying connected with each other. Basically, what this means is that instead of paying for the call, the ringtone of the phone acquires a pre-texted (ignore the pun!) meaning for the receiver, acting sort of like simplistic morse code that is sent according to an evolving sender-receiver feedback loop. Donner writes:

Beeping is simple: a person calls a mobile telephone number and then hangs up before the mobile’s owner can pick up the call. If the beeper’s name and number have been programmed into the recipient’s mobile, then the recipient will see the beeper’s name on the call log as a missed call. If not, the recipient will see only the number of telephone placing the call. In either case, the missed call is intentional; the beeper has sent a signal to the recipient without saying a word or typing a single character. Better yet, sending a beep is nearly free.

As these examples from the popular press suggest, “to beep” and its synonym “to flash” are common practices among the rapidly growing ranks of mobile phone users in sub-Saharan Africa:

• A lighthearted book, called How to Be a Kenyan, was originally published in 1996. The revised 2002 edition includes a new chapter called “of beepers, flashers, and vibrators” (Mutahi, 2002), reflecting the rapid spread of mobiles in Kenya.
• A columnist in Uganda writes: “I was angry with my so-called friends who ‘beep’ me all the time – blackmailing me into calling them back….I can understand someone beeping me once and a while. My problem is that so many Ugandans – from MPs to senior military officers and at least one government minister – have turned beeping into a profession. And, they never seem to realize that if they perennially don’t have “units” or airtime to complete a call, it must be the same for me too” (Pajero, 2004).
• In an article called “That Beeping”, a Tanzanian columnist explains, “beeping is a habit that transcends all social classes,” perhaps because “phones are cheaper to buy than to maintain”. His conversations with college students also suggest that “beeping is a modern fashion to say hi to friends” (Kalagho, 2004).
• On the GhanaHomePage website, expatriate columnist Rodney Nkrumah-Boateng describes his difficulties learning “Efie Nkomo (Flashing Skills)” as a returning visitor to his country. “I got loads of flashes on my phone in the days following my arrival. So many, dear reader, that I almost got blinded by them” (Nkrumah-Boateng, 2004).

As Nkrumah-Boateng succinctly puts it, “it can get confusing, all this flashing businesses”. Not all beeps mean the same thing. Some are requests to call back; some are little signals that the beeper is thinking of the recipient; others convey some pre-negotiated instrumental message, like “I’m done with my work, pick me up”.

Read the full essay HERE. So for us interested in the future media, I think it is sometimes necessary to step offline and outside our e-skins to see how the rest of the world is busy hacking the new techonologies for their purposes. The biggest problem I think that currently exists with the web 2.0 social software mobile etc is that the community that produces the tools and applications is closing in on itself, sharing the language, the desires, the lifestyles and the ways of using technology. Whereas most innovation happens when something unexpected is created and brought to the world – when “the difference that makes a difference” is produced. So the question is: how to break “outside” this? How to see what we are not habitually accustomed to seeing?

Perhaps, we should worry less about the latest trends in technology and backpack through Africa for 6 months and listen to people hacking the tools in order to make life better. Get lost on a train in India, hike in the mountains, and go offline for a month. It is (t)here where the future is being developed in unexpected ways .

Interestingly, as Matti pointed out, there have been several such experiments in the past in the european art scene, but very few have been automated, and none of them have played with Indian content.

Fact is, between Tatu (filmmaker and friend) and myself, we have hours of extremely diverse footage from India – from the weddings, to the stray dogs that live in Mumbai, to Rajasthani life and other such tourism cliches. If there was a way for us to compile and tag the usable bits, and the system is able to map that to a script, and generate thousands of cuts to the video, which I could then do next few cuts with, it could actually turn out to be a very useful tool. We’ll see anyhow.

The plan is to work on it over the next few months on a slow burner, as NewsRoots is on highest priority (with an impending prototype deadline). But once an initial model is ready, we will put it up on both our labs section, and here. In the meantime, we will continue the brainjamming here, while sipping on bottles of Kingfisher and playing with Flash, Ruby-On-Rails and After Effects code.