Skip to page content

Carnegie Mellon University researchers develop new way to 'see' sound using normal cameras


dual shutter photo
Caption courtesy Carnegie Mellon University: Mark Sheinin (left) and Dorian Chan were part of a CMU research team that developed a camera system that can see sound vibrations with such precision that it can capture isolated audio of separate guitars playing at the same time.
Carnegie Mellon University

When played concurrently, sounds — like those from musical instruments — combine together to produce a new noise, but researchers at Carnegie Mellon University say that they have developed a system that's capable of reproducing sounds in isolation of each other, even when those sounds are played in unison.

Mark Sheinin and Dorian Chan, researchers at CMU's Robotics Institute, say that they have developed a new way to see the sound vibrations emitted from an object using a system comprised of ordinary cameras and a laser. According to the researchers, the precision and detail captured by this technology allows for the original sounds of a given object to be reconstructed and then played without any interference of nearby noises.

It's a feat that even the highest-powered microphones can't accomplish, although directional microphones come close in practice. In fact, the system doesn't use any microphones at all to capture the sound waves, relying solely on the visual capture from the cameras and laser, sights that the human eye can't even see.

Sheinin and Chan have since been able to successfully demonstrate the technology's ability to capture the individual sounds of two guitars playing next to one another. The system has also been able to identify and isolate two streams of music coming from two nearby and respective speakers, among other feats.

"Our method can sense multiple points simultaneously and can also support object motions such as the natural motions of a guitar player," Sheinin said in a video explaining the technology.

The 10-page research paper about the technology titled "Dual-Shutter Optical Vibration Sensing" can be found here.


Keep Digging

Profiles


SpotlightMore

Ryan Green, Co-Founder and CEO of Gridwise.
See More
Josh Fabian, CEO and Co-Founder of Metafy outside his their office in Youngwood, PA. their office in Youngwood, PA.
See More
Participants in the Greater Pittsburgh Regional FIRST Robotics Competition on Friday, March 18, 2022, at the Convocation Center at California University of Pennsylvania, in California, Pennsylvania. The competition runs March 16-19th, winners go on to com
See More
With employers searching for a quality workforce and many Kentuckians searching for a new life, there is no better time for employers to expand their fair chance hiring places.
See More

Want to stay ahead of who & what is next? Sent twice a week, the Beat is your definitive look at Pittsburgh’s innovation economy, offering news, analysis & more on the people, companies & ideas driving your city forward. Follow The Beat

Sign Up
)
Presented By