It seems that I missed that Augmented Reality Toolkit all the way until now. It’s ARToolKit and it’s completely OpenSource.
As a matter of fact there are a ton of demos available… HOW could I possibly miss that for so long?
“ARToolKit is a software library for building Augmented Reality (AR) applications. These are applications that involve the overlay of virtual imagery on the real world. For example, in the image to the right a three-dimensional virtual character appears standing on a real card. It can be seen by the user in the head set display they are wearing. When the user moves the card, the virtual character moves with it and appears attached to the real object.
One of the key difficulties in developing Augmented Reality applications is the problem of tracking the users viewpoint. In order to know from what viewpoint to draw the virtual imagery, the application needs to know where the user is looking in the real world.”
Here is a short video demonstration of what you could start with:
…not talking about the things that would be possible if someone had a great idea :-)