Final Project Update

It's been a while, but I've been busy working hard. Here is an update on my progress:

Things that I have done/working:

• The screen is built. I used spandex and pvc pipe to construct a screen that stands 6 ft tall and 6'4" across. The interaction with the screen itself is very nice. The spandex is soft and inviting to touch
• I have the kinect software running smoothly in Visual Studio. After writing some calibration code I am getting a very clean read on the screen (no noise). I had attempted to use Processing originally, but after finally winning the battle to get all of the kinect and opencv libraries that I needed running with Processing on my 64 bit Windows 7 machine, I was dismayed to discover that these libraries used a hacked Kinect driver rather than the official driver released by Microsoft. I am blaming this driver for the terrible lack of quality in the depth information that I was getting from the Kinect in Processing. I decided to go back to the official Windows SDK and learn Direct2D.
• I have a visual application that generates rectangles which are rotated according to how far the screen is being displaced inward. The rectangles are sized according to the radius of the area of the screen that is displaced. This is not the original application that I had planned, but it is visually appealing and fun to interact with. It kind of grew out of my methods for debugging my program to detect areas of pressure on the screen. To detect these pressure points I process the depth image from top left corner to the right and then down. I simply check if the depth at a pixel is greater than zero and then check all of the currently existing "blocks" of pressure/depth which only store the start and max x and y coordinates. I see if this pixel is within a threshold distance of the x or y max then add this pixel to the block by updating the appropriate max value. This is a very fast and efficient way to detect multiple disjoint areas of pressure on the screen. 

Here are some images that my installation has created:

Things that Dont Work or Aren't Finished:

• I am having trouble with a hotspot showing through from the projector. I think my screen is just too thin. You can see straight through the fabric. I thought that offsetting the projector would fix this, but my screen is too big to get the hotspot out of viewing range. I have also tried putting some diffusing materials like wax paper or dryer sheets in front of the projector light but these solutions do not work.
• I originally proposed a fractal exploration application for the screen which I would still like to complete. I have mostly implemented an image sequence exploration application which parses through the frames of the image or video sequence to the depth of the displacement and then plays the video back to the first frame. I plan to make a fractal zoom video using Mandelbulb3D and then use the image sequence viewing technique allow users to interactively explore the fractal

Videos Coming Soon!