The title of the above TED talk about "Sixth Sense" is kind of misleading, the talk is actually about a fantastic natural user interface which is very useful in everyday life. Even though it seems futuristic, it can already be done with current not so expensive equipments. The talk introduced a device that is built from web-cam, pico projector and mobile phone. The web-cam read information from the outside world using bar-code reading, facial recognition, fiducial marker recognition and some kind of image processing. The web-cam also read the user input through finger movements just like what Tom Cruise did in the movie Minority Report, which is what we are currently working in this final year project.
In my speculation, the data from web-cam is then sent to the mobile phone using wireless connection (Bluetooth? Wi-fi?), and then upload to a web application through wireless connection (Wi-fi, 3G etc). I do not believe the mobile phone has the power to process the video. The web application processes the video input (or photos input) and produces a particular web-page which provides information and further interactions in the form of links. This web-page is then projected out to the world in front of the user by using a mirror due to the fact that the projector is more suitable to be pointing down. The introduction of pico projector is really making this device possible. A Malaysia company Aiptek has this kind of pico projector but it is not suitable to build this "Sixthsense" device as it only reads from memory card and has no wireless connection to anything else. Therefore I am really looking forward to mobile phone projector from LG or Samsung which will make it easier to reproduce such device.
Some of the features introduced by the talk are also in the vision of Microsoft, which can be seen in the video below:
More information about Sixthsense.