Project Reality
Zero preparation. We just brought in a couple cameras and 3D-printed some mounts. We had a projector but figured everything out on the day. And this is the coolest project I've ever made. Won Best Overall at MasseyHacks with it.
Virtual reality projected into real space. You throw real balls at a projected game, and the cameras track them with stereoscopic vision. Just watch the video, you'll love it.
The two cameras see the ball from different angles. We find it in each image, triangulate its position, then send the coordinates from the Raspberry Pi to Unity over TCP. Unity uses that position to make the real ball hit things in the game.
Finding the ball was tricky because the projector was shining different colors onto it. We used OpenCV to filter each camera image by hue, saturation, and brightness, with sliders to tune the filters while watching the result. The white patches in this video are what passes through those filters.
The stereo rig was janky looking, but it worked. Here, look.
