Just read this interesting piece on a new synth called ReacTable that has caught Björk‘s eye. It is a light table with a projector and a video camera under it. The video monitors what is going on on top and using the OSC protocol (I think) it sends instructions to sound producing software. The projector sends light to the table, in order to connect and modify the various objects put on the table and moved around to make sounds. Here is a demo movie, and another one with multiple players live in Berlin.
This is part of an exciting field in computing called human interface design. Most of the HID folks are visual/tactile oriented, as in Jeff Han’s light table controller which is a minor YouTube hit. The key point is that these HIDs are multi-touch, so you can use all ten fingers, or multiple people can work on the same surface. Ultimately, I agree with Han that they will replace the monitor/keyboard/mouse configuration. Here is the YouTube presentation from 2006.
You can find a newer demo (and nothing else right now) at his new startup, Perceptive Pixel.
Musicians are on the forefront of this revolution, though I am not sure they know it, or that the computer people have quite realized yet either. In music, the HIDs are called controllers, like the pricey Lemur, which is multi-touch but not touch-sensitive, and the STC-1000 (which I own), which is not multi-touch, but is touch sensitive…and about a tenth of the price of the lemur. What Han and ReacTable are doing that is new is making the surface responsive rather than just an input device. I’ve emailed Han asking about audio applications a year ago or so, but got no reply…guess he was too busy even then.Another place where the arts will influence HIDs is the connection between dance (or more limited motion) and music. My favorite example is Laetitia Sonomi‘s Lady Glove which is a glove full of motion sensors that control synthesizers. It looks sort of like her nervous system has been turned inside out, with the nerves on the outside, which is not a bad metaphor I think.There is also a growing DIY music controller movement, where people hack up toys or build from scratch their own versions of something that will control music. I think what the musicians don’t yet realize is that their search for more sensitive and interactive controllers is something that translates well to other realms.
As always, please check out some of my free music at Way Music.