We’re currently working on a number of Augmented Reality projects which should see the light of day in the New Year. We were also recently approached by prestigious design magazine Wallpaper, to produce a series of Augmented Reality applications for their very first AR issue.

We didn’t have very long to build them and there was a fair bit of overtime involved but the end result was worth it. The piece was directed by Noah Harris at Blinkart with 3D modelling by Glassworks.

So rush out and buy the magazine and then go here to interact with the applications.

Over the past couple of months I’ve had the chance to get in a bit of research and development time with Unity, and one of the areas I have looked at is Augmented Reality. By far the most entertaining experiment so far has been my “AR Driving” demo. It could be said that this particular demo isn’t strictly AR (because it doesn’t overlay graphics on the video stream) but still, it uses AR technology to map the orientation of an AR marker to the steering, acceleration and braking of a car. I recently demoed it in our London office, result: Fun!

The demo works by reading the orientation of a single AR marker printed on an A4 sheet. The sheet is folded so that the marker is visible to the camera on one side, and a steering wheel image is visible on the other side – so the user knows which way up to hold the paper!

The rotation around the Z axis (as in, turning the steering wheel) is mapped to the car steering, although not entirely linearly – in order to make the controls easy to use it required a few tweaks, such as smoothing out the sometimes jittery raw data, and giving bias to a ‘not-quite-dead zone’ around the zero-degrees rotation area to make it easier to drive the car in a straight line.

The acceleration and braking controls are implemented by reading the “tilt” of the marker around the X axis. Holding the marker straight-on to the camera gives the equivalent of a steady gentle press on the accelerator. Tilting the top edge towards the camera pushes down the accelerator, while tilting it away releases the accelerator completely and applies the brakes.

The final result is a fairly intuitive driving control, and most people who had a try in the office seemed to find the car responsive and easy to control on their first try, as the video shows!

Ever since I joined Skive, I’ve been producing 3D games for the web, using Director and Shockwave (not to be confused with the Flash plug-in, which is sometimes referred to as “Shockwave Flash”). I started developing 3D games in 2002, and in the early days, Shockwave’s capabilities were so far ahead of its time that the main consideration was whether the average home user would even have the hardware required to run the game at all.

Some of the 3D Shockwave games we've produced at Skive

Since then however, much to the disappointment and frustration of many Shockwave developers, upgrades to Shockwave’s 3D and game-related capabilities have been almost completely negligible. At the time of writing, Shockwave still has pretty much the same 3D feature set that it had in 2001, and we are now in almost exactly the opposite situation, where most modern computers – even cheap home PCs – have advanced 3D graphic capabilities which the now antiquated Shockwave plug-in simply cannot make full use of.

Adobe claim that their long-term plans include making Director “the preferred environment for games creation”, however until they release a clear roadmap or a significant upgrade in this area, we can only speculate as to what this means.

I recently had the opportunity to research and evaluate some of the modern alternatives for a new project, and of these, “Unity 3D” emerged as a clear winner. I had been keeping an eye on a few of the 3D alternatives over the recent years, but it wasn’t until I actually got stuck in and started developing in Unity 3D that I realised the extent to which it feels ‘alive’ as a tool. The community is buzzing with new ideas and talent, the company is responsive and easily approachable, and the engine’s capabilities are modern and expansive.

Unity 3D - The Editor

As well as the obvious graphical advantages that come with a modern 3D engine, there are many other features which throw Director’s shortcomings into stark contrast. Here are just a few:

• Supports C#, a modern professional programming language. In addition, you can work on the code in Visual Studio, which sweetens things even more.
• Very active forums, with many Unity engineers and product specialists contributing regularly.
• A user feedback system where ideas and feature requests can be added and voted on. Members of the technical team actually comment on these ideas, and some of them do end up getting implemented!
• A public roadmap. They tell you what they’re working on, and roughly when it’s due for release.
• New communities are growing around the technology. People are tweeting and blogging about Unity. There are typically 40-80 users in the IRC channel at any given time (including regular Unity staff). By contrast, the community around Director feels as though it has been dwindling since around 2004, and of those remaining, a high percentage are old-timers.

So now, having completed my first (rather large) Unity 3D project, I’ve come away very impressed with Unity – with both the product itself, and with the company and community that comes with it! And on that note, I’m hoping to dedicate a little more time to such things as blogging about my experiences, experiments and research over the next few months.

Since my AR Business Card went viral (over 300,000 views to date) both myself and Skive have had a great deal of attention. Skive are currently working on, pitching and persuing around 7 AR flash projects for some big brands.

Myself, several other Actionscript developers and our 3D artist have been looking into FLARToolkit (the Flash AR code library) and are fully up to speed in this area. We are working on a few more viral videos and concept stuff and have produced a few examples, including our very own logo.

In addition to this we are dedicating time to research a variety of other AR approaches including colour and face detection and languages such as C++ and Objective C (iPhone).

So watch this space over the next couple of months, we should be releasing some fascinating, functional and playful AR applications.

Skive and Multi-touch Technology

A few weeks ago, Skive were invited to the Microsoft offices for a presentation on their Surface technology. Here we had a chance to get a hands-on with their intriguing multi-touch technology which prompted some investigation into touch devices in general.

Getting your head around ‘blob tracking’

The technology behind these multi-touch devices is surprisingly simply to understand. Unlike pressure sensitive devices found in phones, the software monitors images received from a camera or cameras looking for changes in brightness and contrast. Placing a hand on a diffused screen above the camera forms a visible shadow / pressure point that can be seen from underneath. By applying filters and thresholds to this image ‘blobs’ are generated that can be tracked as the hand moves.

Refinement of this detection is scalable to cost, improvements are made with infra-red technology, controlled diffusion and professional cameras. However, it is actually possible to make one of these devices yourself with nothing more than a cardboard box, picture frame and a webcam.

Building your own

There are a number of tutorials on the web than can show you how to do this (links below), for my box I followed the great tutorial on the NUI group forums. For the hardware I chose a webcam that performs well in low-lighting conditions and had an acceptable frame rate of 30fps. Construction of the box is relatively simple, center the camera and ensuring the boxes edges are level is straight-forward, using the correct paper for diffusion is essential as A4 white is too thick to get an accurate result so I used grease-proof paper (although tracing paper is also a good option).

On the software side I used the opensource program from the NUI group called community core vision or tBeta. This program allows you to monitor input from a number of sources, including pre-recorded video in case you don’t have a device available. All input is extensively configurable, allowing you to adjust for lighting conditions and other environmental variables in real time.

CCV stands as an intermediary layer between your multi-touch applications, providing a socket server that broadcasts TUIO data about the blobs that it is tracking.

This data can be received by any technology capable of connecting to this socket allowing complete freedom over the language you choose to program in. The installation package for tBeta comes with a number of Flash .swf examples, and a more examples for other languages can be found on their website, forums and wiki.

Additional links:

» Hardware tutorial for creating your own multi-touch cardboard box
» Community Core Vision (aka tBeta or “The Beta”)
» Oxfiphone project/library, openframeworks expansion for the iPhone
» Videos tagged on vimeo – click here and here

Duncan and Matthew attended Flash in the Can in Toronto in April - an intense four days of training, conferences and networking.

By the end both were fully loaded with new information, business cards and inspiration. The intellectual resilience to this sensory overload was telling, and on the final night Duncan let his hair down and partied within the inner sanctum of the FITC Leet, resembling a character from one of the sessions - Interactive Story Telling – Zos World;

Matthew sat in his hotel room wistfully tweeting about the background ambience of the Toronto Tamil protests – go figure.

“A simple, featherweight headset, a 10′ x 10′ x 10′ white room, and $600,000 worth of projector and computer equipment, combined with the smarts of the folks at Eon Reality, results in one insanely real experience”.

We love seeing new technologies, like this project by Eon Reality, being used to create immersive digital experiences.

Amazing 3D immersion technology from IDEO Labs on Vimeo.