MIT researchers have turned a smartphone LiDAR sensor into a system that can see around corners. (Image Credit: Aaron Young/MIT Media Lab)
One might think it’s impossible to see around corners without special equipment. But MIT researchers have changed that by using smartphone-based LiDAR sensors that cost under $100. Their technique, motion-induced aperture sampling, uses this hardware to reconstruct hidden 3D objects and track moving targets around corners. According to the team, this could be used for self-driving cars to detect other vehicles, cyclists, or pedestrians. For robotics, it can help robots move around obstacles in their path.
Non-line-of-sight (NLOS) imaging requires extremely precise, high-cost LiDARs ($50,000+) with thorough configuration and calibration. Consumer-grade LiDAR sensors use low-power lasers, and that means the images they capture tend to be noisy. These are also low resolution and may produce unclear photos if the camera and target objects move.
Rather than relying on a single image, the team combined data from multiple frames to reveal hidden objects. They drew inspiration from burst photo capture and radar imaging, which combine multiple inputs over time to produce better results. When the researchers developed algorithms to merge data across those measurements, the hidden signals started emerging.
In their experiments, the researchers used a smartphone LiDAR system with approximately 100 pixels. Each one has a laser emitter combined with a single-photon detector. With this system, they mapped hidden objects in 3D and followed their movement when they were known shapes. It also determined the LiDAR sensor’s location by using those hidden objects as reference points. This technique worked without special calibration and is useful for robots in difficult-to-navigate areas.
However, this system isn’t a camera that creates images of hidden scenes. So far, it only pulls out sparse shapes and hidden clues from extremely faint signals. It’s still not the same crisp, megapixel-quality people can see on their phones. The system works best when object shape and motion barely change between frames, as researchers can then combine many weak measurements into one clearer signal. This assumption may not always be true, however, as postures can change, objects may disappear from view, and the LiDAR sensor may move unexpectedly. Those conditions make data harder to understand.
Next, the researchers want to make the system less reliant on current assumptions. Stronger physical models, signal processing, and machine learning may help it handle more complex motion and changing scenes. That may make around-the-corner sensing more reliable.
They also note that a different hardware design could improve the system. Modern consumer LiDAR is for normal depth sensing. Optimizing future sensors could detect visible and hidden scenes. Doing so can improve sensitivity, resolution, scan patterns, or optics for better NLOS performance.
Have a story tip? Message me here at element14.