At the first DDDNorth, I did a session on Kinect and the Kinect SDK. One of the question I get asked is “can it track my fingers?” Well, the SDK doesnt let you do this – yet. However, because you have access to the raw video streams, theres no reason why you cant do it yourself.
Microsoft Research have taken a Kinect, and using the reflection on a piece of glass from a normal monitor, and managed to recreate a HoloDeck environment, right on your desktop – Called HoloDesk.
It lets you manipulate 3D objects in real time, and allows intrgration with real world objects. Its much better to see than explain, so watch this video.