Our latest project looks at the different ways we can use emerging digital technology to help understand the environmental changes taking place today especially in rural Asia and Africa.  That is, if proper ecological management and knowledge around climate change, pollution, biodiversity loss, urbanisation etc will be one of the biggest challenges of the future, how can then our unique style of research, technological prototyping and art experiments help us prepare for what is to come. The project that we are launching – at this point tentatively called The Internet of Nature (or Project ION) – will therefore combine classical research, technology and experimental art around what we believe to be some of the key topics and themes of the emerging future.  How could such work then help us predict and prepare for the changes that will be reality, say, 2, 5, or 10 years from now?  What is needed?

We have already started work around these important questions and hope to expand to more areas with increased time and resources:

1. BOOK CHAPTER/CONFERENCE: We are next presenting at the “Between mainstream and the fringe: environmental activism in a globalised world” conference on December 9th and 10, 2010 in Delhi.  This conference is co-organised by University of Heidelberg in collaboration with Centre for Culture, Media and Governance, Jamia Milia Islamia University in New Delhi.  We will also write a book chapter for the conference book, which will outline explicitly some of the theoretical and practical issues that we are dealing with around environment, research and digital media.

2. MCO2.  We have been already working for the past year on research and a technological prototype around a mobile-based solution for climate change  targeted especially towards the millions of smallholder farmers in Asia and Africa.  The project is done in collaboration with environmental scientists and farmers in Ethiopia. The following description gives a good overview of the kind of work/research we are developing:

CHASING THE LONG TAIL OF CLIMATE CHANGE

The core idea behind mCo2 is simple: you provide us with the details of a tree or a plant – we calculate the amount of carbon dioxide tied into its biomass, its change over time and its current market value. On top of this simple concept, all kinds of services from agroforestry management tools to carbon offset trade platforms to even environmental gardening communities can be built.

Current approaches to climate change so far, we believe, have been too focused on reducing the harmful effects of consumption, our so-called carbon footprint. This, however, neglects the fact that most people in the world are not only consumers – they are also producers. This is to say, they are involved, in one way or another, in the production of plants and trees into which carbon dioxide is tied in complex ways. This applies as much to the millions of smallholder farmers in rural Asia and Africa as it does to hobbyist gardeners in US and Europe. Given this fact, then, how could we reach out to this ‘long tail’ of climate change: the millions and millions of people involved in the production of biomass for their livelihoods and who are now increasingly connected by the rapid spread of mobile phones and the Internet? Moreover, how could we best encourage the positive effects such activities can have both on the environment but, as importantly, for the livelihoods of the people involved?

Our solution is simple. By combining innovative web & mobile-based technology with sophisticated science, we are able to develop new tools that make this paradigm shift possible. The low carbon economy of the future will be as much production-based as it will be consumption-based – our mission is to make this possible. We are currently testing the demo version of mCo2 together with a project working with farmers in rural Ethiopia, of which you can see the first version here being tested out.

Please read more about this here at New York Times or at our prototype website HERE.

3. INTERNET OF NATURE – THE BOOK. We are eventually also planning to write an experimental theory-technology-art book around these themes.  The planned book would of course contain some of the the philosophical-theoretical foundations around the concept of “the Internet of Nature” but also examples of the technological and art experiments that help us best understand this “archeology of the future.”

For the past six months Matti and Surpreet have been providing some BCG expertise (as the digital artist & the technology poet) to an innovative UK-based community art project in Bradford.  The project  Sounds Like Graffiti combined audio plays, rap music, theatre, drama, video, digital storytelling and mobile technology to come up with a new innovative way of doing location-based art and guerilla broadcasting in public spaces using the potential of new and accessible and cheap digital technology.   See some of the coverage for the project HERE and HERE.

Another description of the project reads as follows:

Since February, 2010, we, a group of international artists from the UK, Finland and India have been working together with young people in Bradford, UK, to create short sounds plays audiences can listen to via their mobile phones in one of the city’s most historic landmarks. This cross-media project – titled Sounds Like Graffiti – uses a diverse range of methods from participatory theatre, digital art and video, rap and electronic music to produce two minute audio plays that reflect the lives of the youth in the city notorious for its crime, poverty and ethnic tension. The final result, a 35-minute play, aims to help the young people of Bradford’s most deprived communities to express themselves through creative writing, audio production, mobile telephony and audiovisual forms as well as to celebrate a new generation of young artists emerging from the city.

This art project will finally culminate in a weekend-long locative media exhibition in one of the oldest landmarks in the city, Lister Park, where the members of the public will be able to walk through the park and tune into these short episodes using their mobile phones (as well as through an online multimedia exhibition). The story follows the adventures of one girl sent out to find her wayward brother. On her quest, she walks through the park and gets lost. The journey takes her through a series of fantastical adventures mixing the story of Odysseus with elements of the tough life young people face in Bradford today. The public walking through the park will be able to move closely together with the story through the different locations while listening to these adventures being played out either via their mobile phones of with the help of volunteer tour guides armed with

mp3 players and speakers.

So in our never-ending quest for the Idea, we have been recently throwing around some ideas around AI, aesthetics and creativity. To understand this particularly strange one, however, I probably will need to provide a few words of background to the wider project that we are interested in. I will try to keep it simple. Basically, the problematic/question that I am working around has to do with the “image of thought” in today’s digital cultures. One of the key tropes we are seeing emerging today is that the universe is made of patterns; or “abstract machines” as I prefer calling them after the French philosopher Deleuze. Without getting into unnecessary theoretical complexity here, what this means can perhaps best be seen inflected and explained in two ways:

The first is that “things” (in technical language “identities” or “forms”) are the emergent outcomes of complex repetition of patterns through which complexity if formed. These can then be reverse engineered and computationally modelled so that, say, a fractal algorithm can be used to model complex patterns in nature. This is especially common today in some of the 3D and visual compositing software that we use today and all kinds of experiments with genetic algorithms and cellular automata etc – simple rules, complex outcomes.

The second is that human intelligence itself is made of, to a large degree, of such repetitions. This idea is rather much more elusive than the first but is one of the key presuppositions behind artificial intelligence research. Here the key question is that what algorithms could be used to model the way humans think and thus be used to guide machines to perform complex tasks. The philosophical implications of this are even more profound than getting a robot to recognize faces or clean a non-linear toilet bowl. That is, if human intelligence is, in fact, highly programmable, what then defines humans from machines? This goes two ways: machines-as-humans and humans-as-machines. In other words, AI defines rationality a certain way with certain presupposition of what logic, thinking and consciousness are and how they can be pragmatically simulated in computers. But as importantly, if we look at the concept of rationality and how it has been historically constructed, this has always presupposes a certain “image of thought” that has excluded all that would not fit into the sphere of rationality (intuitions, insanity, madness, illogic, spontaneity, absurdity ….). So how would we then understand the blurred boundaries of man and computer (as intelligent forms, which neither technically speaking are) and the human-computer assemblage that is making the old notions of rationality/humanity perhaps increasingly difficult to defend? Humans as (programmed?) repetitions: computers as programmed repetitions: natural intelligence: artificial intelligence: natural stupidity: artificial stupidity …

Anyway, all this probably seems rather abstract here. However, what was a concrete outcome of this philosophical babble was a project called AML (or what we like to tentatively call Aesthetics Mark-Up Language) that we are now currently trying to get our head around and develop. Similar to the more philosophical questions above, the idea here would be to experiment with how we detect patterns in certain visual styles such as certain genres in film etc. These patters could be then transposed/translated to other instances so that, say – yes, however preposterous this may sound – a Bollywood Film could edit a Hollywood film!

To perhaps see more what we mean by this, please find below an excerpt of some of the behind-the-scenes work-in-progress. The email conversation explains the idea better than I could re-write.

In film theory, for instance, the following elements are often talked about in the semiotics of the filmic image (well it is more complex but bear with me for the time being …)

1) Images;
2) Phonetics = speech;
3) Noise = background noise etc;
4) Text
5) Music.

For our purposes, we will probably have to stick to images for now. Tracking classical elements of film, in the beginning, is far too complex to get started with. Such as depth of field, montage / contrast etc. So what could be easily tracked to get started?

Let’s start with images. We could start off with the following variables:

- movement (speed of movement = speed of change in pixels?) This could be later used to analyze some rhythm of change.

- brightness and contrast (how would this be tracked = the relationship or average of pixels in any given location on the video?)

This could also later be use to analyze things such as harmony of composition, direction of lines in the mise en scene, etc. We would have to come up with a set of principles from art history and composition and see how these could be determined in the screen etc?

- color range (this would probably have to be RGB values in the image itself). This would probably move us into the realm of things such as monochromatic color schemes, bright colors, harmonious colors, contrasting / oppositional color … ie to use some notion of color theory to provide patterns in certain styles of video etc. I’ve
studied this in high school so will be fun to revisit some principles of classical painting.

So I suspect what we need to do is set up a very simple experiment / structure in place that can be developed and extended depending on need. In other words, we need to develop … AML (Aesthetics Meta Language) … a basic language structure that would describe what the variables are within any analyzed video. This language, I suspect, could be then developed into the interface between the language of aesthetics and the computer. Something like this:

//AML: “DEBBIE DOES DALLAS”

<contrast>
<high>134</high>
<low>12</low>
<average>58</average>
<mean>22</mean>
</contrast>

You get the point. The interesting thing about such an approach would be that we would be developing an entire syntactics of the aesthetics-computer interface that could be taken to any given direction we want. And then, of course, transposed back to how we would edit any clips or piece of media that we choose to run through the framework.

So how would this relate to the aesthetics machine? Basically, determining before hand the parameters for aesthetics is impossible. In film theory, there are some ideas such as structuralist semiotics that try to describe the language of different films but – as you know – aesthetics is notoriously difficult to pinpoint, especially when we are stuck with pixel-level pattern recognition on the computer. So what the aesthetics machine could be is the extension of the AML meta-language that would emerge from experimenting with as much visual material as possible. Say, we make AML public and get film-buffs and media-freaks to go through the entire ouevre of, say, German Porn genre to see what patterns emerge. These are then described in the wider structure of the AML language that we start developing So we get:

//AML: “DEBBIE DOES DALLAS” AESTHETIC STRUCTURE

<crossfade>
<number>12</number>
<duration>145</duration>
</crossface>

<contrast>
<high>134</high>
<low>12</low>
<average>58</average>
<mean>22</mean>
</contrast>

etc etc. Perhaps we could even open this language up so that some elements could be done analogically by volunteers (ie Amazon Turk Model) to further build the language beyond what is possible through pattern recognition. As this language develops, then the aesthetics machine will then merely be the instantiation and translation of these variables to other media material.

So if this approach would be the best, we need to develop / conceptualise three things:

1) The pattern recognition engine to get started experimentation with
simple data;

2) The syntactic structure that would describe any given visual material and that could be populated both by computers as well as analog humans;

3) The basic experimentations of how the AML language could be then
used to edit / re-mix some other media element.

So one reason why we are interested in Processing is exactly that it is the ideal platform for such experiments to emerge. In any case, watch the space here as our ideas eventually develop and we start getting the first concrete experiments up.

One of the older projects that we worked with in 2007-2008 was the prototype for a citizen journalism platform called Newsverse aimed at emerging economies especially in India and Africa.   While the platform/development is not active anymore and we have moved onto newer more contemporary projects, this project was important for BCG as it showed us for the first time clearly the  enormous potential of combining innovative  research and technological development as a method for working with contemporary digital media.  The project also spawned a book chapter on emerging digital cultures in India as well as many other fruitful collaborations.  You can read about the background to Newsverse below or see the website archive HERE which explains a bit more about what we were doing:

The idea for NewsVerse emerged over a series of discussions about the Indian news media and the challenges it currently faces. Two of us, Somnath Batabyal and Matti Pohjonen, had recently conducted our doctoral research on the news production practices in the country’s television, newspaper and new media industries while Somnath had also had a successful career for over decade as a journalist in India. Soumyadeep Paul, in turn, is a developer and a software architect who had worked for a number of high profile new media and web 2.0 companies in the US and India and had been a key figure in the nascent blogging and video blogging communities there.

The discussions focused on what we perceived to be the biggest problem facing the Indian news environment today: the homogeneity of content, despite an increasingly competitive and experimental market. While new newspapers and television channels were being launched at an unprecedented pace, the diversity of voices represented was nonetheless shrinking. With each news channel targeting the richer segments of the audience because of their spending power, the scope for a plurality was reducing. Ironically, this lack of diversity is taking place amidst the backdrop of the unprecedented possibilities offered by emerging new media technologies. India has currently 190 million mobile phone users and the numbers are, according to some estimates, growing by more than a 100 million each year. Internet and broadband penetration rates are also rapidly growing and for the first time reaching beyond the old enclaves of metropolitan centres touching people previously not affected by the global information revolution.

So the question that we were interested in was: how could these two parallel developments be reconciled: the shrinking space for public opinion amidst the possibilities for communication offered by new media forms and technologies. We find this to be one of the most important social challenges facing India, but, more broadly, all emerging economies in Asia and Africa today: that is, how to use new media technologies to democratize the production of knowledge as the Internet and mobile use gains new users in previously unaffected areas and demographics.

NewsVerse therefore emerged out of desire to bridge these two: the need for a diversity of voices offered by citizen journalists and the increasing interest amongst the mainstream media and organizations for such activity. In specific, it does this by providing a novel structure for citizen journalists to produce news and communicate in a social network environment while collaborating with the mainstream media and organizations to provide financial incentives to do so. Thus, it aims at promoting the formation of a more pluralistic news environment but, as importantly, it aims at getting mainstream news channels to actively promote and benefit such citizen journalist activity by forming close collaborations between citizen journalists and the more established mainstream media and organizations.

Interestingly, many of  the ideas that we were experimenting with at the time have only now become mainstream – 4 years later – with services such as Demotix becoming more commonplace and the boundary between citizen and mainstream media becoming increasingly blurry.