Mark Sheinin (left) and Dorian Chan helped develop the dual-camera system that can capture isolated audio of two guitars playing and reproduce the sound. (Image Credit: Carnegie Mellon University)
Carnegie Mellon University researchers developed a camera system specifically designed to reproduce the music from an instrument in a band or orchestra by picking up the sound vibrations. The team's concept involves using two cameras along with a laser to detect high-speed, low-amplitude vibrations. The system then takes advantage of the vibrations to regenerate sound, recording isolated audio without relying on a microphone.
"We've invented a new way to see sound," said Mark Sheinin, a post-doctoral research associate at the Illumination and Imaging Laboratory (ILIM). "It's a new type of camera system, a new imaging device, that is able to see something invisible to the naked eye."
The researchers tested the camera system's capability to detect sound vibrations and reproduce sound at high quality. So they captured "isolated audio of separate guitars "playing simultaneously and speakers that played different music. Afterward, the team ran an analysis on a tuning fork's vibrations and used a Doritos bag's vibrations to capture the speaker's sound.
In this project, the team used traditional cameras, which are less costly and generate higher quality recordings than the high-speed types used in previous research. Both cameras capture an object's vibration from movement, including a guitar played by a musician, and detect several sounds from multiple points.
The dual-camera system functions through speckle pattern difference analysis via images taken from a rolling shutter and global shutter. Afterward, an algorithm calculates the speckle pattern differences from both videos and reproduces the sound by converting the differences into vibrations. The speckle pattern is produced by targeting the vibration-producing object's surface with a laser. Then, the speckle pattern changes during the object's surface vibration. A rolling shutter rapidly scans an image from top to bottom before creating the image by stacking one row of pixels atop another. Meanwhile, a global shutter captures an image in one go.
Sound engineers can use this system to observe instruments producing music without interference from the other ensemble, allowing them to fine-tune the sound. The dual-camera system could also apply to manufacturers for monitoring machines that vibrate in a factory so they can determine if maintenance is needed.
Have a story tip? Message me at: http://twitter.com/Cabe_Atwell