Affordances and Constraints: Expression as an Instrument
October 22, 2006
A few years ago I was mangaing a Nesta Project called Muzantiks. During this time I booked a developer who was just out of his MA course at Middlesex, the unfortunately name DIM (Design for Interactive Multimedia). Thor had been working with Enrike developing interactive musical instruments and were about to go their own ways back to their native countries. I suggested they formed themselves as a artists group and carry on with the projects- and so ixi-software was born.
Since then they lectured, toured, performed around the world, demonstrating the possibilities of screen based interaction for music.
Originally, the applications were sampled based tools;’ now they produce interfaces for high powered Open Source audio engines such as MAXMSP, PureData and Supercollider, so incorporating a vast array of synthesis and sampling. The interfaces they design and produce are perhaps best described as non linear composition tools (as seen here with SpinDrum), lending themselves to live performances more than studio based tools. The genius of them, aside from the design of the GUI’s that step away from traditional modes of address when producing music, is that their applications are loosely coupled with the audio engine through a protocol called OSC. Open Sound Control is a way to transfer information, rather much like MIDI, but with the ability to transmit abstracts not literal information. Less about telling a system to play D#, rather play harmonies around D#…
Thor has been talking a lot about affordance recently; Affordance is how something is identified as useful in a certain situation, sometimes outside the expected or designed use of the object. He’s now released a paper as part of his Phd and is a fantastic accumulation of thinking and doing in the realm of interactive media. The paper, titled Affordances and Constraints in Screen-Based Musical Instruments, is available here. Here’s a quick insight on the nature of the paper:-
“As opposed to acoustic instruments, the screen-based digital instruments are not of physical material so all mappings from a GUI element to the sound can be arbitrarily designed. This arbitrariness is even more apparent as there is hardly a tradition for creating such instruments. The metaphors we use in ixi software are new in a musical context and deliberately have no musical reference. (such as depicting keyboards, strings, notes, etc) The decision to exclude metaphors from the world of music comes from the aim to get away from the cultural constraints that are connected to the historical instruments or their parts.”
Faris from Naked Communications has also been talking about Affordance his blog - in relationship to marketing consumer propositions.
I think there is something very potent here that has yet to be factored into the communications industry. Messages are usual so refined so that there is no ambiguity and thus very little affordance. Imagine producing messages that allow a vast array of communications thus extending the value of the work. Brands, when polyphonic, allow such affordance.
Video is the hot subject of internet commerce, yet still it’s a linear model of communications. The use of tagging does open up it’s use and ability to shift context and this model is exemplified in the Chris Andersons book: The LongTail.
As media producers comprehend there is no singular destination for their work: the pda, the laptop, the television, the ipod are all nodes in the communication framework – and so by making not only the portability of the communication part of the production affordance, but also the meaning of the communication to be as affording as possible too.
One way the media industries are looking to expand the affordance of their productions and that is to extract meaning from video streams. This is a buggy workaround to a problem that is best solved through design of the media and not through trying to hack media objects to acquire their affordance. Here’s a short video interview with Suranga Chandratillake of Blinkx. Blinkx as he says in the video was set up to solve the in video search problem, and interestingly he acknowledge this is not a technical solution, but a creative one.
Context advertising thrives on the ‘refindability’ of media – thus the value of the media is squared to the retrieval-ness. If the retrieval-ness was actually driven by downstream usability, that is, the ability to incorporate the communication with other media to expand the value of both communications, the audiences ability to construct relevant narratives latches their desire to engage in dialogue with the publisher.
This is affordance via licence – if media objects have an open licencing or reuse, the value of the message persists.
This brings the attention back to the HCI; the way the audience interacts with media objects such as audio and video. Much of the success of the relentless bombardment of web2.0 applications is based around doing something very simply which in turn give buckets of affordance when you mix these services. I’m referring to web API’s, where by the extraction of data from various sources can be ‘mixed’ to produce new meaning and use of the various data sources. Dapper and SalesForce are examples at either end of the enterprise spectrum. Here’s a trilogy of video clips featuring Jeff Hunter of Electronic Arts talking about their use of SalesForce’s services. Considering, it’s been said to me, that Electronic Arts put the EA in sweatshop (ahem), this is one company that a) understands interactive media b) expects a lot of affordance from the user experience. c) understands talent affords the company it’s ability to develop better products.
In terms of the use within the communications businesses, lets turn the model around; think less about what you want to say, but what you dont want to say. This is about building in restrictions of the use of the media – some form of protection about who the media cant be missused, missuntersood. This is aligned to the thinking about the role of Digital Rights Management (DRM). This annoys the heck out of most people who want portablity and freedom to use media.
So, how do the web2.0 services deal with this affordance. Well, there is the XML-RPC protocol. This enables one service to access meaning from another service without accessing private data. It’s a gateway, not an open door. Rob has done a lovely little write up on this recently.
Consider iTunes|iPod combo – it’s a gateway to the Longtail of music, but it’s not the source of expression as their campaigns may lead you to believe. The itunes|ipod service, indeed the Apple business model is to attach you to their gateway, not for you to be a value added network to their network. By all means promote their services (the ubiqitous white headphones being key), but don’t interfere with the source of expression, namely, sharing their clients copyrighted material. Remain an individual and rock on…
[iPod — Silhouette (Love Train theme by Wolfmother)]
As we see with SalesForce, the concept of the mashable web is about to saturate the enterprise media platforms, under the heading of Service-Oriented Business Applications (SOBAs). Jason Bloomberg, who was to present at this, describes the role of the enterprise mashup as:-
“For a mashup to be an enterprise mashup in that it addresses a particular business problem, tight coupling between provider and consumer software would be a serious concern. Most of today’s mashups, however, care little about loose coupling. Mashups that meet business needs, therefore, will require SOA, and the SOA infrastructure necessary to guarantee loose coupling. Without that loose coupling, mashups are little more than toys from the enterprise perspective. “
So lets think about Service Oriented Media Applications – software solutions that generate media propositions, that afford the user experience to be expressive. If you’re thinking this is User Generated Content, then you’re not imagining hard enough. A SOMA should inherently have a licence to create, and any such creations should be reused, reworked and help other users to experiment and be expressive. A brand that develops SOMA’s, becomes the ‘source of expression’, not the framework of expression – and if you manage the source of a SOMA, you’ll have a loyal user base which starts a whole range of traceable dialogues. These dialogues are markets and the brand becomes a maker of instruments not melodies.
Affordance implies a freedom to experiment, to find expression where none was expected. Where services and communication defer the audience to participate in expression, we can expect to lose their attention – and that’s something you cant afford.