Progress

I cleaned up the login screen a bit and have gotten some of the geolocation dataworking:

http://www.resoundproject.com/development/

I began work on the session browser:

http://www.resoundproject.com/development/server.php?action=sessionbrowser

And I’ve begun work on creating a generic collaborative keyboard for the sessions:

http://www.resoundproject.com/development/server.php?action=entersession

I’m currently playing with Auxlib, a javascript library which does alot of thing, but has a pretty interesting synth library.

http://clouddev.org/auxlib

Posted in: Uncategorized

Geolocation

Added an automatic geolocation system for determining information based on ip address:

Thanks to: http://www.united-coders.com/christian-harms/free-ip-location-script

Posted in: Uncategorized

User System

Using this tutorial, http://php.about.com/od/finishedphp1/ss/php_login_code.htm, I have an extremely proto user login system up and running.

http://www.resoundproject.com/development/

Posted in: Uncategorized

Proposal – Approved 10/10!

---

TABLE OF CONTENTS

---

Abstract

Using existing collaborative music and locative art projects as my starting point, I will explore the changing notion of location within collaborative music environments as they move from physical spaces into digital spaces. From this research, I will design a collaborative online sound tool which reconfigures the idea of physical location in relation to collaborative networked music spaces.

---

Introduction

Traditionally, location has played an important role in the creation of sound. From the perfectly sloped vaulted ceilings of the Sydney Opera House, to the carpet covered sound proofing of a rock band rehearsal space, location is something that is often thought of as having a direct influence over the quality of a sound being produced. Sometimes this can be attributed to the acoustics of a given space, or the ability of a space to reverberate naturally. Other times, this reflects something more undefinable, a sort of ambiance or mood that is produced by a specific location in relation to the sound being produced.  The time of day, the weather, or the history of a space all have the ability to affect our perception of a location. The shape of a space, the path we took to the space, the tonality of the instruments, the material of the walls, and the surrounding area of a space can all affect our perception as well.

Sydney Opera House Band Rehearsal Space

In the 1990’s, the Free Networks Movement suggested that “ubiquitous Internet access would change our relationship with place by overlaying a second virtual world over the physical one.” The Internet would not only be everywhere, but it would be colliding with the physical world in a transparent way. Today we find mobile devices with Internet access to be widespread and data about location and space is just a couple button clicks away at any given time. Geo-taggers and locative media artists have augmented our physical world with a virtual one, full of excess information, real-time data, and digital graffiti. Suddenly, our physical space and location has been extended to the very virtual layer which the Free Networks Movement predicted in the 1990’s.

This augmented locative experience has branched out, affecting everything from the ways in which we travel from place to place, to the ways in which we study and the ways in which we get our news. But in this world of excess data, how does physical location become redefined in relation to sound when sound finds its very means of creation being uprooted simultaneously? With the introduction of electronic music, we found almost immediately that our creative spaces went from practice halls to headphones. With the introduction of easily accessible digital reverberation and compression tools, the natural sound quality of a space became less important, and in some cases became more of a nuisance than something that is desirable. With the rise of collaborative networked music tools, we find location being pushed farther into the background and disregarded in the experience of musical creation and performance.

This is not to say that physical location has been completely forgotten, it is just being reconfigured for digital space.  Definitions of space have gone from the physical and cultural characteristics of a given location to more infrastructural reflections of space. Physical addresses have been replaced by ip addresses, while reverberation is replaced by network latency. Our locations have become both physical and digital in a sense. 10 years ago, Mike Liebhold of The Institute of The Future (IFF) predicted that in the next decade, “context-aware computing will emerge as the third great wave of modern digital technology”. A decade later, the IFF’s prediction has come to fruition, finding a world in which mobile cell phones outnumber home computers, participatory sensing is used in everyday life, and participatory data is now widely distributed. While we once could only see information about what was directly in front of us, or readily available to us, today we have access to infinite amounts of data in relation to a given location. We can easily access information pertaining to the cultural background of a space, the shape of a space, the path we took to arrive at the space, and not to mention endless amounts of information about the area surrounding a space. This poses the question, how can this data be used to reflect the given characteristics of a space? How can this data be used to re-establish the notion of physical location in the creation of music?

Within the domain of networked music, I find two areas of focus to be of particular interest when discussing location, the first being the area of networked collaborative music creation. A quick search on the Internet reveals endless amounts of solutions aimed at solving the problem of logistical issues in the creation of music. NINJAM supports performers playing to a synchronized loop over a network, cleverly placing everything to a beat to playfully cover up latency issues. Indaba and eJamming coordinate the social networking of musical projects, allowing for the simple creation of an online band, and then allowing for composing and jamming to be done while online. While solutions like these tackle a problem of logistics and location, they do actually very little to address the actual definition of the physical space itself. These tools have helped to solve a problem of logistics, creating an online collaborative space, while disregarding the notion of physical location altogether.

My other area of focus is the locative musical synthesizer; projects which sonify infrastructural data into musical compositions. With the rise of the computer as a sound creator, the field of sonification emerged. Sonification can be defined as “the transformation of data relations into perceived relations in an acoustic signal for the purposes of facilitating communication or interpretation.” Clouds, wind, sounds, and virtually any data source all become the raw material from which sound relations are created. Virtually any data that is available can be used to create a sound counterpart via sonification. Mori, by Ken Goldberg, produces piano melodies and simple graphs based on seismographic data. The Cloud Harp project sonifies video of clouds from a given location into harp music. Sonic City sonifies our interactions and our movement throughout an urban space into sound, creating a personalized soundtrack based on our relation to the city.  These projects take the reconfiguring of location and poetically redisplay it in a way that is both emotional and meaningful.

It is between these two areas of focus that we find a gray and somewhat unexplored area: the combining of networked collaborative sound creation with the notion of locative sound production and sound sonification. Sites such as Pachube.com, Every Block, WeatherNet, Flight Tracker, Geese in Space, and endless others provide us with a constant stream of real-time data that is easily accessible. Given the affordances of the Internet and its seemingly endless supply of real-time locative data, I propose it would be possible to harness this data in a way that would both guide and restrict the collaborative creation of music. Hammet, in his article Beyond Locative media suggests that “locative media offers a conceptual framework by which to examine certain technological assemblages and their potential social impacts.” In this project, my technical assemblage becomes this uncontrollable, endless, and easily accessible stream of data that has been given to us. It would be possible to create sound with its emphasis on location, across multiple locations in a way that has not previous been possible. Using this type of locative data, we could create an instrument which absorbs this influx of data and reflects it in it’s timbre. What might it be like to play music from Atlanta with a friend from Chicago? What would it be like if the sounds produced were a direct reflection of the mood of those two cities, pulling from traffic conditions, temperature, crime statistics, and the actions of the person performing the music? What would it be like to choose to virtually relocate and be given the ability to play the mood of a neighboring city? Our paths, our interactions, the acoustics of the surrounding area could all be possible catalysts in the creation of sound.

Concept

The focus of this project will be both sonification and collaboration concentrating on space and sound. The system will create a unique instrument for each user based on information they select from their location. An example of this information might be data pulled from an RSS feed from the National Weather Service, such as temperature, barometric pressure, or weather conditions. Data could also be pulled from participatory data services such as Pachube, such as a reading of all CPU temperatures within a 10 miles radius. A site such as traffic.com could be used to reflect the current mood of the traffic conditions in a given area. This data will then be used to create an instrument module that the timbre is modified in real-time as it is played via a keyboard in relation to the data. The user has control over the pitch of the instrument, while working within the creative restraints imposed by the locative data. Within a collaborative setting, users can play together over the Internet, combining the unique qualities of sound, which is related to their location, with their own personal performances. Users will also have the option to play through modules based on other locations and other collaborators. The system will  have a recording element to it, in which a composition can be recorded, and then played back through the same system, creating unique effects related to the real-time data of the location each time it is replayed.

Possible Real-time Data Sources

• Pachube.com – Real-time participatory sensing
• Traffic.com – Real-time national traffic information
• EPA – Real-time environmental information
• NASA – Real-time satellite information related to environment and weather
• The National Weather service – Real-time weather information

Theory and Research

In her article, Breaking Out: The Trip Back, Helen Thorington discuses some early efforts of creating networked music applications over the Internet. From the earliest efforts in creating networked music, something that became immediately apparent is that the space created stood in opposition to that which was created by a more traditional collaborative music experience. Musicians can easily sit in a room together and pass ideas back and forth quickly and efficiently, but it was immediately apparent that the networked environment was full of flaws, having problems with latency and unpredictable computation times; in short, the Internet in its infancy was not a place in which collaborative music creation could easily take place in a one to one relationship similar to its physical world counterpart. Fortunately, as Thorington points out, we are living in a world where Umberto Eco’s notions of the open work prevails, from the performance works of the Fluxus movement to the sounds-capes of composer John Cage, we have become more accustomed in the last 30 years to works which have an openness and element of chance to them that lies beyond a personal interpretation of the work, but instead in the uniqueness of each individual performance of a work.

It is within this non-closed framing of sound that networked music really came into its own. These networking flaws were used to an advantage in works such as The Grimm, which openly exploited issues with latency and computational difference between systems. Issues that resulted in unpredictable behaviors were used to purposely and intentionally produce a piece which changes upon every viewing. Networked music seemed ideal within the environment of jamming, in which timing might not be as much of an issue. But we are now in an age in which we are slowly moving beyond the limitations of the Internet. This is not to say that  latency will eventually be an entirely cured phenomena. Hub musician Jesse Gilbert believes latency will always be present in the field of collaborative networked music creation, stating “no matter how low it can be taken a network interaction can never be said to be real-time in the way that a traditional concert can, and frankly I’m not sure that’s the point”. Traditional musical instruments all restrict their players in their own unique way, and from this statement, Gilbert views latency as nothing more than one of the expected restrictions of creating music over the Internet. It is not that this environment has become latency free, but that the creators of music within this environment have come to expect latency and except it as an inherit part of creating music under these conditions. Gilbert focused his later work on this question: “How do we move beyond the amorphous Internet ‘jam session’ by engaging strategies that acknowledge the challenges of Internet-based musical interactions, yet are not limited by them?”. It is today that I believe this question has a new translation for new problems. We are slowly moving beyond the limitations of the Internet, having turned its flaws to our advantage for the purposes of creating a collaborative musical environment, now how can we exploit the environment within which the sound is create itself? How can we create sound which takes advantage of the affordances of the Internet, as opposed to creatively dealing with the problems of it? In short, we have reached the point in which the question is no longer how can we deal with the Internet and its inherit flaws, but how can we move beyond this and use the Internet to our advantage?
Speaking on the rising practice of locative media, Marc Tuters and Kazys Varnelis state “[s]ince its inception, then, locative media’s practitioners have claimed an avant-garde position, insisting not only that their work is capable of creating a paradigmatic shift in the art world, but also that it can reconfigure our everyday life as well by renewing our sense of place in the world”. This reconfiguring of everyday life can be viewed as one of the many affordances of the Internet, especially in relation to physical space. The physical space surrounds the act of networked music, from the area in which a networked piece is accessed, to the maze of cables, network paths, and routers a message is delivered over. But what has become of this physical space in relation to the act itself? What has become of  our ’sense of place in the world’ when the notion of place in the context of networked music has become more and more disregarded, and simply presented as a problem which needs to solved? The reconfiguring I am suggesting is the side by side placement of infrastructural data which represents physical location.

We now find one open end of networked music which  involves speed, latency, and computational times, coming to its final stages. Will the collaborative networked environment ever truly be free of latency issues? Probably not, but in the last 10 years we have seen somewhat of a technical improvement, and even more or an overall acceptance of the Internet and its flaws. It is within this acceptance that we find the newly open world of real-time networked data and locative information. Within my project, I intend to extend this notion of Eco’s open work within music to the unpredictable source of real-time sensing, participatory sensing, and location based data which the Internet has afforded us. Utilizing this as the grounds for the creation of an open based sound art project, I will explore possible answers to the questions which Gilbert has posed, and how we can use the newly allotted affordances of an Internet full of endless encyclopedic data to move beyond the amorphous jam session.

Related Work

The Hub: [http://hub.artifact.com/]

The Hub is one of the earliest experiments in collaborative networked music creation. The group consists of John Bischoff, Tim Perkis, Chris Brown, Scot Gresham-Lancaster, Mark Trayle and Phil Stone, and grew out of the League of Automatic Music Composers.

Mori: [http://memento.ieor.berkeley.edu/]

Sensing minute movements of the Hayward Fault in California, Seismographic information is translated to data, and then into visual information and sound. Location, and change within that location, play a key part in the creation of piano notes and graphs which reflect not only our relationship to the earth, but how we interpret that information.

Indaba: [http://www.indabamusic.com/]

Indaba Music promotes collaborative networked music endeavors through the creation of online communities and social networks. Indaba helps create the connections between users wanting to participate in network music collaboration. Musical endeavors are project based, and Indaba provides a common space for connecting with likeminded musicians for a specific project.

Sonic City: [http://www.tii.se/reform/projects/pps/soniccity/index.html]

Sonic City explores the notion of space and our everyday interactions with that space. Wearable electronics and participatory sensing are used to create a personalized soundscape which reflects the space, ambiance, mood, events, behaviors, and interactions with our surroundings.

Heat and the Heartbeat of the City: [http://www.turbulence.org/Works/heat/]

In this project, temperature data and statistics from New York City are sonified into the sound of a heartbeat, reflecting gradual changes that point towards the effects of global warming.

Cloud Harp Project: [http://www.turbulence.org/blog/archives/001624.html]

The Cloud Harp takes direct locative data related to cloud patterns and sonifies it into sequences which is played back on a harp. The cloud harp explores the direct relationship between our location and sound, and how data collected directly from around ourselves can have a direct relationship to sound.

Technical Description

Resound will be a web-based application which will provide the framework for the creation of online location based collaborative music. The project will be both a website and a stand-alone application. The website will consist of a basic user login for the purpose of communication between collaborators, and will be used for the collecting of locative data based on each user. The musical collaboration and performance will either take place in a cross-platform standalone desktop interface, a web browser, or will be built on top of an existing platform such as Reason.

The website will be built using HTML, CSS, Flash, Flex, Javascript, XML, PHP, and mySQL. The back-end of the site will run on Apache, and all code will be cross-browser compatible. AJAX will be used for all server/client communications, data will be stored using mySQL, and all graphics will be created using Adobe Photoshop and Adobe Illustrator.

If the application is stand-alone, then the back-end will use Flash Media Server for the creation of collaborative Flash based applications. The project will rely heavily on external sites for the collection of extra locative data, including weather, traffic, temperature, and climate information. This information will be drawn from sites such as Pachube.com, traffic.com, the national weather service, and others using AJAX calls and parsing XML that is retrieved.

• XML – XML will be used for data sent between PHP and Javascript
• Javascript/AJAX – Javascript will be used for server communication calls
• PHP – PHP scripting will be used for server side scripting
• mySQL – mySQL will be used for storage of data on the server side
• HTML/CSS – All the visual website layout will be done using/CSS
• Adobe Air – Adobe Air will be used to create an environment which operates across platforms and between local and networked data and hardware sources
• Flash Media Server – This server will be used for creating collaborative flash applications
• MAX MSP – Possible interface for creating synthesizer module
• Processing/JAVA – Possible interface for creating synthesizer module

---

System Diagram

Timeline For Project

• 10/5 – 10/25 – Experimenting with different synthesizer platforms
• 10/26 – 11/8 – Building/Mockup/Prototyping initial sound modules with data input
• 11/2 – 11/8 – Prototyping initial website environment
• 11/9 – 11/15 – Prototyping initial collaboration environment
• 11/16 – 11/22 – Refining design of sound modules for website testing
• 11/23 – 12/13 – Programming site and testing connections
• 12/14 – 12/20 – Testing sound modules within the collaborative connections
• 12/21 – 12/27 – Testing website and collaborative environment
• 12/28 – 1/3 – Site Launch
• 1/4 – 4/24 – Site Testing/Reiterations
• 4/25 – Presentation

Scenarios

Scenario 1

An art gallery is putting together a locative media project involving musical collaboration between multiple cities. The media project will involve 4 performers from 4 different cities, playing in collaboration from their respective galleries. Each gallery space will be divided up into four quarters, and each quarter will be a virtual space representing the location of the other gallery spaces.

On the day of the performance, all 4 performers log in and create an account with the system previous to their performances. They create a session and all log into the session. A locative music module is created for each of the locations and performers. Using the session controls, the session creator is able to create a session in which all the collaborators inputs are run through each of the collaborators locative music modules. Each of the individual locations are routed out through 4 individual speakers in the 4 corners of each gallery space.

Throughout the performance, the same collaborative music is heard, and can be heard as a reflection of the four locations by navigating around the 4 corners of the gallery space.

---

Scenario 2

Lydia is an artist who is working on a multi-location installation piece. The piece consists of sculptures that reflects the sounds of their own creation, and are accompanied by a soundtrack which is intended to reflect the mood of the environment in which they were created.

The sculptures were all created in the same location, but on different dates, different times of days, and during different weather conditions. Composing a simple melody, Lydia logs into the Resound project website during the time of the creation of each sculpture. Lydia loads a prewritten melody into the local music module, and records the melody during the creation of each sculpture. The melody is consistent throughout each recording, but the tonality of the instrument itself is a reflection of the time of day, the weather, and the local traffic conditions that were present during the creation of the sculptures. Each sculpture serves as a reflection of both its creation and the mood of its surroundings during the time of its creations.

Scenario 3

Jeremy is a musician who has collaborated many times in performance and composition with his friend Mikey. Unfortunately, since moving to New York, this has become an impossibility. While both have been able to email projects back and forth, the musical experience is lacking the sound and ambiance that was once achieved while playing music together in a practice space setting.

Jeremy and Mikey decide to turn to the Internet to find a collaborative networked music solution that could maybe help recreate the experience of playing together in the same space. They both log into the Resound Project, entering information related to their simultaneous locations. This information is stored in a database, along with their ip addresses, and other information which relates to there individual locations.

On the server side, a profile is created for each user, along which a locative music module which is relative to the users location.

Both Jeremy and Mikey log into the website, and are presented with a personal musical synthesizer which is created on basis of their locations.

From a friends list, Jeremy and Mikey are able to create a new collaborative session or join a pre-existing one. Mikey creates a new session, and Jeremy joins. Both using a keyboard, Jeremy and Mikey can collaborate on musical jamming or recording a new composition, creating unique sounds which are relative to each of their locations.

Scenario 4

Thomas has written a musical composition which is to be performed in a number of different locations. He wants the composition to maintain some semblance of itself each time it is performed, while achieving a unique quality to the tonality of the instruments each time the piece is performed as well. However, Thomas wants this unique tonality to somehow be a reflection of the location in which the music is performed.

Logging into The Resound Project, Thomas chooses the option Quick Location Instrument creation. Based on ip address information, the system is able to determine an estimated location of where Thomas is. Using this data, the system creates an on-the-fly locative music module: a unique instrument that can be played solo or in a collaborative networked setting. At each performance, this unique instrument is used to reflect the settings in which it is performed.

References

The Poetics of the Open Work, Umberto Eco, 1981, http://interactive.usc.edu/members/akratky/W3_Open_Work.pdf.

The Death of The Author, Roland Barthes, 1977, http://www.vahidnab.com/author.pdf

Theory of the Avant-garde, Peter Bürger, 1984, 3. The Negation of the Autonomy of Art by the Avant-Garde’, University of Minnesota Press,  pp.47-54.

Notes on the Elimination of the Audience, Alan Kaprow, 1966

Relational Aesthetic, Nicolas Bourriaud, 2002, http://wiki.mediamind.org/images/3/38/Bourriaud.pdf

Social Aesthetics, L Larson, 2006, Participation, MIT Press, Cambridge

The Ethnography of Infrastructure, Susan Lee Star, 1999, http://abs.sagepub.com/cgi/reprint/43/3/377

Sonic City: The Urban Environment as a Musical Interface, 2003, Lalya Gaye, http://portal.acm.org/citation.cfm?id=1085741

Beyond Locative Media,  Marc Tuters and Kazyz Varnelis, 2005, http://networkedpublics.org/locative_media/beyond_locative_media

The Production of Space, Henri Lefebvre, 1947, http://www.notbored.org/space.html

The Practice of Everyday Life, de Certeau, 2005, http://www.digitalcenter.org/pdf/Center-for-the-Digital-Future-2005-Highlights.pdf.

The Social Logic of Space, M Certeau and S Rendal, 1984, http://wiki.mediamind.org/images/6/60/Decerteau.pdf

The Social Logic of Space, B Hillier and J Hanson, 1984

1000 Plateaus, Deleuze, G Deleuze, F Guattari – Berlin, Merve, 1992

Sonification Report, Gregory Kramer, Bruce Walker, Terri Bonebright; Perry Cook; John Flowers; Nadine Miner; John Neuhoff, Year Unknown, http://www.icad.org/websiteV2.0/References/nsf.html

Sound The Net, Jesse Gilbert Interview, Year Unknown, http://www.fictive.org/cmr/appendix/gilbert.html

Posted in: Uncategorized