Gesture Control of Spatialization

This project aims to develop novel compositional and technological methods for the advanced use of the multidimensional nature of auditory space in music composition. Specifically, the project involves the examination, design and construction of novel methods for gesture control of sound source position, sound source parameters and room model parameters within a sound spatialization system.

Research being performed within this project includes:

• Motion capture of performer gestures to determine which aspects of the gestures may be used to control spatialization
• Development of novel gestural controllers for manipulation of sound source and room model parameters
• Design of systems to unobtrusively capture 'accompanist gestures' which performers make during performance
• Evaluation of existing controllers as tools for gesture controlled spatialization
• Examination of novel methods and metaphors for manipulating sound sources in a spatialization system

Other aspects of the project which are being developed by our research partners include:

• The design and development of a 24-channel audio virtual environment (AVE)
• Examination of the perception of spatial audio
• Compositional aspects and uses of gesture-controlled spatialization

Along with the research aspects of this project, there is a significant musical component. A number of pieces are being written, including works both for solo cello and also for a larger ensemble. The first of these performances took place as part of the 4th International Conference on Enactive Interfaces, and will be followed by larger performances in Montreal, QC, Canada as part of the MusiMarch08 event and again later in 2008 in BC, Canada.

Further information is available from the project wiki.

• Mark T. Marshall, Joseph Malloch and Marcelo M. Wanderley. “Non-Conscious Control of Sound Spatialization..” In Proceedings of the ENACTIVE'07 Conference, Grenoble, France, Nov. 2007.
• Joseph Malloch, Stephen Sinclair, and Marcelo M. Wanderley. “From controller to sound: Tools for collaborative development of digital musical instruments.” In Proceedings of the 2007 International Computer Music Conference, Copenhagen, Denmark.
• Mark T. Marshall, Joseph Malloch and Marcelo M. Wanderley. “Gesture Controlled Sound Spatialization for Live Musical Performance.” (abstract) Presented at the 2007 International Gesture Workshop, Lisbon, Portugal.
• M.T. Marshall, N. Peters, A.R. Jensenius, J. Boissinot, M.M. Wanderley and J. Braasch, “On the Development of a System for the Gesture Control of Spatialization.”, In: Proc. of the International Computer Music Conference 2006 (ICMC06), pp. 360-266, New Orleans, U.S.A., 2006.