Build music with blocks: Audio d-touch

Researchers at the University of Southampton have developed a new way to generate music and control computers.

"Grab a block and add a base beat, turn a block to speed up the high hat and we have a new way to generate music through controlling the computer," said Dr Enrico Costanza at the University's ECS - Electronics and Computer Science, who is launching Audio d-touch (Thursday 25 August).

Audio d-touch, which is based on Dr Costanza's research into tangible user interfaces, or TUIs, gives physical control in the immaterial world of computers. It uses a standard computer and a web cam. Through using simple computer vision techniques, physical blocks are tracked on a printed board. The position of the blocks then determines how the computer samples and reproduces sound.

"As more of our world moves into the electronic; records to mp3s, books to eBooks, we loose the satisfying richness of touching physical objects like paper and drumsticks," said Dr Costanza.

"Our Audio d-touch system allows people to set up and use tangible interfaces in their own home, office or recording studio, or wherever else they like," said Dr Costanza. This is the first time that anyone has developed a free application like this."

Audio d-touch is more than just for play: TUIs are an alternative to virtual worlds. Human-Computer Interaction researchers are investigating ways to move away from the online, purely digital world and rediscover the richness of our sense of touch.

Dr Costanza has developed Audio d-touch over several years, "Our aim is to advance the field by gaining insight into how tangible interfaces can be used in the real world. We are keen to have more people download audio d-touch and give us feedback so that we can improve it."

All that is needed is a regular computer equipped with a web-cam and a printer. The user creates physical interactive objects and attaches printed visual markers recognized by Audio d-touch. The software platform is open and can be extended for applications beyond music synthesis.