Skive and Multi-touch Technology

A few weeks ago, Skive were invited to the Microsoft offices for a presentation on their Surface technology. Here we had a chance to get a hands-on with their intriguing multi-touch technology which prompted some investigation into touch devices in general.

Getting your head around ‘blob tracking’

The technology behind these multi-touch devices is surprisingly simply to understand. Unlike pressure sensitive devices found in phones, the software monitors images received from a camera or cameras looking for changes in brightness and contrast. Placing a hand on a diffused screen above the camera forms a visible shadow / pressure point that can be seen from underneath. By applying filters and thresholds to this image ‘blobs’ are generated that can be tracked as the hand moves.

Refinement of this detection is scalable to cost, improvements are made with infra-red technology, controlled diffusion and professional cameras. However, it is actually possible to make one of these devices yourself with nothing more than a cardboard box, picture frame and a webcam.

Building your own

There are a number of tutorials on the web than can show you how to do this (links below), for my box I followed the great tutorial on the NUI group forums. For the hardware I chose a webcam that performs well in low-lighting conditions and had an acceptable frame rate of 30fps. Construction of the box is relatively simple, center the camera and ensuring the boxes edges are level is straight-forward, using the correct paper for diffusion is essential as A4 white is too thick to get an accurate result so I used grease-proof paper (although tracing paper is also a good option).

On the software side I used the opensource program from the NUI group called community core vision or tBeta. This program allows you to monitor input from a number of sources, including pre-recorded video in case you don’t have a device available. All input is extensively configurable, allowing you to adjust for lighting conditions and other environmental variables in real time.

CCV stands as an intermediary layer between your multi-touch applications, providing a socket server that broadcasts TUIO data about the blobs that it is tracking.

This data can be received by any technology capable of connecting to this socket allowing complete freedom over the language you choose to program in. The installation package for tBeta comes with a number of Flash .swf examples, and a more examples for other languages can be found on their website, forums and wiki.

Additional links:

» Hardware tutorial for creating your own multi-touch cardboard box
» Community Core Vision (aka tBeta or “The Beta”)
» Oxfiphone project/library, openframeworks expansion for the iPhone
» Videos tagged on vimeo – click here and here

Duncan and Matthew attended Flash in the Can in Toronto in April - an intense four days of training, conferences and networking.

By the end both were fully loaded with new information, business cards and inspiration. The intellectual resilience to this sensory overload was telling, and on the final night Duncan let his hair down and partied within the inner sanctum of the FITC Leet, resembling a character from one of the sessions - Interactive Story Telling – Zos World;

Matthew sat in his hotel room wistfully tweeting about the background ambience of the Toronto Tamil protests – go figure.