Tactosonix

Participants: Bruno Angeles
Marcelo M. Wanderley (supervisor) 		The initial TactoSonix FTIR setup, with paper as a projection surface.
Funding: N/A
Project Type: Master's thesis.
Time Period: Sept. 2009–2012 (Windows installer & documentation available.)

Project Description

TactoSonix is a multitouch environment for musical expression. It consists of a hardware interface that was designed and built by the author and an Industrial Design consultant, as well as software that is being developed as part of the author's M.A. thesis research.

The TactoSonix hardware stands out from most other multitouch devices in that it supports two methods of implementing multitouch technology: Frustrated Total Internal Reflection and Rear Diffused Illumination. As such, it is an excellent platform for prototyping ideas about tangible objects, and also offers a more robust finger tracking method in its FTIR configuration.

Software

Screenshots

The following screenshots show the TactoSonix software in the environment based on the metaphor MAKING MUSIC IS COOKING.

The first screen of the TactoSonix cooking metaphor environment A focus position resulting in an approximately equal contribution of each audio channel The unwrapped Stacked Half-Pie Menu showing a choice of audio tracks. The first menu represents (from left to right) lead instruments, bass, and percussions. The second menu represents (from left to right) jazz drums, hip hop drums, funk drums, and country drums. Dragging a node around (the red square) displays visual feedback about the associated audio track's volume, low-pass cuto frequency, and loop length. Dragging a node around (the red hexagon) displays visual feedback about the associated audio track's volume and low-pass cuto frequency, looping being disabled due to the node being outside the looping zone of the stovetop. The visual feedback showing that the node is not looping at 1, 2, 4, or 8 beats, is that no crosshair is visible in the cooking pot.

Hardware

The TactoSonix hardware consists of a DIY multi-touch table that supports both FTIR and RDI technologies.

A view of the power switch and toggleThe monitor input and USB webcam output of the tableThe two tabletops (RDI on the left, FTIR on the right)One side of the tabletop allows the plexiglass component to be removed. It is held in place by screws.
The TactoSonix hardware and a JazzMutant Lemur device, in a concert setting. Note IDMIL's dimple software running on the multitouch table. The initial TactoSonix FTIR setup, with paper as a projection surface.Part of the circuit inside the TactoSonix hardware.
The image processing steps performed by Community Core Vision with TactoSonix using FTIR technologyAn inside view of the DIY table, showing the power source, projector and (broken, to be replaced) mirror, webcam, fan, and electronic circuit.
Part of the FTIR circuit showing an infrared diode and its accompanying resistor. The screws on the right implement a simple contact connection with the bottom part of the table to aliment the switches without dangling wires.A full view of the tabletop. The ripples of the silicon compliant surface are visible, since this plexiglass cover is the one for the FTIR setup.
The silicon-covered RDI illuminatorThe TactoSonix hardware at IDMIL. The floating plastic on the tabletop is protection from dirty fingers.
, , ,