According to media reports, the University of Michigan has developed a new real-time 3D motion tracking system that can replace imaging system technologies such as lidar and cameras in autonomous driving applications.

This technology combines transparent light detectors with advanced neural networks and is expected to be used in the fields of automated manufacturing, biomedical imaging and autonomous driving. Zhaohui Zhong, an associate professor of electronic and computer engineering at the University of Michigan, and his team have developed a transparent, nanoscale, and highly sensitive graphene photodetector. The imaging system takes advantage of this detector. Since graphene is very sensitive to light, it is enough to produce images that can be reconstructed by computational imaging. These photodetectors are stacked together to form a compact system, each layer will focus on a different focal plane, so as to achieve 3D imaging.

In addition to 3D imaging, the team also handles real-time motion tracking. For this, they need a way to determine the location and direction of the tracked object. The researchers said that typical methods include lidar systems and light field cameras, but both have obvious limitations.

ZhenXu, a doctoral student in electrical and computer engineering, created the optical device and worked with the team to enable neural networks to interpret location information. The neural network is trained to search for specific objects in the entire scene, and then only focus on the objects of interest. “Although training the neural network takes time, once the training is completed, when the camera sees a certain scene, it can be within a few milliseconds. Give the answer within.'

The team successfully tracked a beam of light and used a stack of two 4×4 pixel graphene photodetector arrays to track the ladybug. In addition, the researchers also proved that the technology is scalable. They believe that for some practical applications, only 4000 pixels are required, and for more applications, only a 400×600 pixel array is required. Although imaging system technology can be used with other materials, one advantage of graphene is that it does not require artificial lighting and is environmentally friendly.