Hearing your favorite song inevitably makes you whip out your proverbial ‘air guitar’ for those blistering solo’s heard in Kansas’s “Dust in the wind.” However playing an actual air guitar has just been that; playing air but hearing no sound of your own. Until now that is, with the help of Kinect. With the ever present Kinect mods it was only a matter of time before someone decided to make a ‘real’ air instrument and proceed to play in front of the ever increasing ‘air crowds’ and the man who did it is named Chris O’shea. After his first attempt in 2007 using a stereo camera that didn’t work too well for him, Chris’s next chance came with Microsoft’s release of the Kinect. Using openFrameworks and openCV for image processing (that he wrote in c++); he then used the ofxKinect add-on in conjunction with libfreenect driver for use on his Mac that helps with target location placement of hands and ‘air equipment’. Chris describes how this works, “First it thresholds the scene to find a person, then uses a histogram to get the most likely depth of a person in the scene. Then any pixels closer than the person to the camera are possible hands. It also uses contour extremity finding on the person blob to look for hands in situations where your hand is at the same depth as your body. It only works if you are facing the camera front on. Then it uses one hand as the neck of the guitar, drawing a virtual line from the neck through the person centroid to create the guitar line. The other hand is tracked to see if it passes through this line, strumming the guitar. The neck hand position controls the chord.” Keep on rockin' in the free world Chris!
Eavesdropper