Conducting Sound in Space

Intuitive måder til at styre lydens bevægelse i rummet via en laptop computer.

v/ professor Wayne Siegel

“Conducting Sound in Space” er et kunstnerisk forskningsprojekt, hvor professor i elektronisk musik, Wayne Siegel, har undersøgt og udviklet intuitive måder til at styre lydens bevægelse i rummet ved en live performance.

Siegel har udviklet software, som benytter det indbyggede webcam i en laptop computer som en bevægelsessensor. Kamerabilledet er opdelt i en matrix med 12 felter. Hvert felt er forbundet med en lyd, således at solisten kan styre lydene ved at bevæge sine hænder i de 12 felter. Herudover blev der eksperimenteret med forskellige bevægelsessensorer til at styre lydenes placering i rummet.

Systemet blev afprøvet tre forskellige steder: ZKM’s Klangdom i Karlsruhe, Den Sorte Diamant i København og Symfonisk Sal i Musikhuset Aarhus. Et nyt værk for solo-performer og motion-tracking system blev udviklet og afprøvet.

Læs professor Wayne Siegels egen introduktion til projektet (på engelsk) nedenfor.

En mere uddybende artikel af Siegel om projektet og om baggrunden for det findes her (på engelsk).

Projekt presentation

The goal of this project was to explore the potential of electronic music that combines production, performance and diffusion into a single integrated creative process. In pursuing this goal I hoped to develop intuitive ways of controlling sound in space within the context of live performance. The project was part of an artistic research project supported by the Danish Ministry of Culture.

In my previous work I have used camerabased motion-tracking to control sound. The image from a webcam is divided into a matrix of twelve rectangles. The amount of movement in each  rectangle is calculated in software by comparing each video frame with the previous video frame. Using this mapping algorithm I can control twelve sounds individually and dynamically by moving my hands in front of my laptop computer: the more I move in each rectangle, the more sound is heard. Multiple sounds can be controlled by moving in multiple rectangles.

The idea of Conducting Sound in Space was to expand on this performance technique to allow real-time control of the positions of the sounds within the concert space using a large multi-speaker  setup.

My first experiments in working with live control of diffusion took place at ZKM in Karlsruhe, Germany in September 2015. The Klangdom at ZKM is a small concert hall equipped with a digital  mixer and 47 independent speakers.

Intuitive control of sound diffusion is a complex issue. It can be difficult to imagine, realize or even perceive multiple audio sources moving in various patterns and at various speeds at the same time. Controlling complex spatial movement in a live situation can be a great challenge.

I used two Hot Hand USB controllers to control panning. The Hot Hand is a commercially available MIDI-controller manufactured by Source Audio. The controller consists of two units: 1) a 3-axis (X, Y, Z) accelerometer embedded in a plastic finger ring with a built-in bluetooth transmitter and a built-in battery and 2) a separate receiver unit, designed to be connected to the USB-port of a computer.

At ZKM I developed the concept of rotational panning. Instead of thinking in terms of panning individual sound sources between speakers, I imagined the whole room rotating left and right, or  back and forth. When I rotated my right hand clockwise all of the speaker positions would rotate to the right, as if I was floating in a fixed position while the whole room rotated clockwise. When I raised my left hand all of the speaker positions would rotate backwards, as if the whole room was rotating backwards.

After my visit to ZKM I had an opportunity to conduct further experiments at the Black Diamond in Copenhagen, as composer in residence at the Royal Library. The permanent 12-channel sound system at the Black Diamond has 12 main speakers and 2 subwoofers hidden in the ceilings on three levels of the main foyer or atrium. The speaker setup is asymmetrical in correspondence with the asymmetrical architecture. The 12-channel setup consists of three trapezoids on three different floors or levels. When performing I stand on a bridge on the second level overlooking the harbor.

Finally I experimented using a 12-channel speaker set-up at Symphony Hall in Aarhus, a hall with a seating capacity of 1,200 that was acoustically designed for symphonic music. There is no  permanent sound diffusion system in the hall, so I was at liberty to place the 12 speakers wherever I wished. Speakers were placed on stage (narrow stereo pair plus subwoofers) above/behind the stage (wide stereo pair), two on each side of the audience on the ground floor, two on the side balconies and two on the rear balcony.

Based on my experiments I found that using two accelerometers to control rotational panning could be combined with camerabased motion tracking to provide a flexible and intuitive interface for controlling live diffusion. I chose this configuration for a new work for solo performer and motion-tracking system. This work, entitled Ritual, employs both camera-based motion tracking using the webcam of a laptop computer as well as a pair of accelerometers, one worn on each hand.

The composition Ritual (Beta) was tested in a concert situation on February 18, 2016 in Aarhus Symphony Hall. The final version of Ritual is scheduled for premiere at the Black Diamond in Copenhagen on April 30, 2016 as part of the Music of the Spheres festival.

For detailed information and video examples see under “research” at www.waynesiegel.dk