Bio-inspired device captures images by mimicking human eye

Drawing inspiration from nature, Penn State scientists have developed a new device that produces images by mimicking the red, green and blue photoreceptors and the neural network found in human eyes.

The research findings, reported in Science Advances, are fundamental breakthroughs in realizing perovskite narrowband photodetection devices — from materials synthesis to device design to systems innovation. The research was supported by seven grants from the U.S. National Science Foundation, listed at the end of this article.

"We borrowed a design from nature — our retinas contain cone cells that are sensitive to red, green and blue light and a neural network that starts processing what we are seeing even before the information is transmitted to our brain," said Penn State researcher Kai Wang.

To achieve this in an artificial device, the scientists created a new sensor array from narrowband perovskite photodetectors, which mimic our cone cells, and connected it to a neuromorphic algorithm, which mimics our neural network, to process the information and produce high-fidelity images.

Photodetectors convert light energy into electrical signals and are essential for cameras and many other optical technologies. Narrowband photodetectors can focus on individual parts of the light spectrum, such as the reds, greens and blues that make up visible light, the scientists said.

"In this work, we found a novel way to design perovskite material that is sensitive to only one wavelength of light," Wang said. "We created three different perovskite materials, and they are designed in a way that they can only be sensitive to red, green or blue colors."

The new technology may represent a way around using filters found in modern cameras that lower resolution and increase cost and manufacturing complexity.  Because the researchers used perovskite materials, the new devices generate power as they absorb light, potentially opening the door to battery-free camera technology.

"The device structure is similar to solar cells that use light to generate electricity," said Luyao Zheng, a researcher at Penn State. "Once you shine a light on it, it will generate a current. So, like our eyes, we don't need to apply energy to capture this information from light."

This research could trigger further developments in artificial retina biotechnology. Devices based on this technology could someday replace dead or damaged cells in the eyes to restore vision, according to the scientists.

Because the algorithm mimics the neural network in human retinas, the findings could provide new insights into the importance of these neural networks to vision.  

NSF supported the research through these grants:

Directorate for Computer and Information Science and Engineering 1718474; Office of International Science and Engineering 1829573; Directorate for STEM Education 1723687, 1821766, 2113839.