Sound being recorded by MIT researchers via footage of vibration of nearby objects (via MIT)
Maybe you heard this already... It seems that the far-fetched gadgets from James Bond films are becoming a reality. Novel technology created by MIT researchers, Microsoft and Adobe can convert the subtle vibrations of an object in a room into sound waves in order to spy on conversations. While they hope this technology may lead to a variety of unexpected inventions, they are currently relishing the cool factor.
Footage is captured by focusing on an object within the room. The footage is high-speed, typically between 2,000 and 6,000 frames per second in order to get the best quality audio. The highspeed enables the footage to capture the minuscule vibrations which can be seen as a result of a nearby conversation or music. The footage passes through an algorithm that converts the recorded object into audio, which is spookily accurate in comparison with the original. In one of their experiments, they were able to accurately interpret speech from behind soundproof glass, 15 feet away.
The masterminds behind this are MIT researchers Abe Davis, Frédo Durand, Bill Freeman, and Neal Wadhwa. Also joining them are Michael Rubinstein from Microsoft and Gautham Mysore from Adobe. To capture the subtle vibrations in objects, they had to employ a unique method of capturing the micro-vibrations that are imperceptible to the naked eye. The vibrations of objects in a room move at about a tenth of a micrometer, according to the research team.
Hence, in order to capture these micrometers, the team had to capture 5 thousandths of a pixel. In order to do this, they take close-up footage and note the change of a single pixel's color in order to note changes in video smaller than a pixel.
The camera footage utilizes various camera filters, which note variations in pixel's colors, over time, in a variety of scales, positions, and orientations. An algorithm then interprets the data from each filter in order to reconstruct the movement of the object over time. It is at this point that clear boundaries can be seen from the variations in pixel color, which notes minute movements.
Their YouTube demonstrates their tests of the technology using a variety of filmed objects and different type of sound (including speech and music). While the technology is definitely not infallible and slightly seems to alter the timbre of the original, it is still quite accurate. For the most part, there seems to be occasional dropped sounds, or notes. Overall, it is quite eerie how accurate this technology already is considering they want to continue to develop it. I guess now the NSA will have even less trouble spying on the public using surveillance video.
Their findings will be published as a paper for the Siggraph computer graphics conference. Hopefully someone will catch the wind of this innovation at the conference and develop new applications based upon this technology. While spying is an obvious application, Davis is hopeful about the possibilities this technology opens up in materials science. The properties of an object alter the type of vibrations produced. The group wonders what application this can have in determining the material properties of an object judging solely by the manner in which it reacts to sound vibrations.
C
See more news at: