December 1, 2022

Breakthrough in machine vision: this device can see millions of colors

An interdisciplinary team of Northeastern researchers has built a device that can recognize “millions of colors” using new artificial intelligence techniques – a massive step, they say, in the field of computer vision, a space highly specialized with broad applications for a range of technologies.

The machine, which the researchers call “A-Eye”, is able to analyze and process colors much more accurately than existing machines, according to an article detailing the research published in Materials Today. The ability of machines to sense or “see” color is an increasingly important feature as industry and society become more largely automated, says Swastik Kar, associate professor of physics at Northeastern and co-author of the research.

“In the world of automation, shapes and colors are the most commonly used elements by which a machine can recognize objects,” says Kar.

Northeastern Associate Professor of Physics Swastik Kar. Photo by Alyssa Stone/Northeastern University

The breakthrough is twofold. The researchers were able to design a two-dimensional material whose special quantum properties, when embedded in an optical window used to let light into the machine, can process a rich diversity of colors with “very high precision” – something practitioners on the ground did not. could achieve before.

Additionally, A-Eye is able to “recognize and accurately reproduce ‘seen’ colors with no deviation from their original spectrum” thanks also to machine learning algorithms developed by a team of AI researchers, led by Sarah Ostadabbas, an assistant professor of electrical and computer engineering at Northeastern. The project is the result of a unique collaboration between Northeastern Quantum Materials and Augmented Cognition Laboratories.

The essence of technological discovery centers on the quantum and optical properties of the class of materials, called transition metal dichalcogenides. Researchers have long hailed the unique materials as having “virtually limitless potential”, with numerous “electronic, optoelectronic, sensing and energy storage applications”.

“It’s about what happens to light when it passes through quantum matter,” Kar explains. “When we grow these materials on a certain surface and then let the light through, what comes out of that other end, when it hits a sensor, is an electrical signal that [Ostadabbas’s] group can treat as data.

When it comes to machine vision, there are many industrial applications for this research related to things like autonomous vehicles, agricultural sorting, and remote satellite imagery, Kar says.

“Color is used as one of the main components to recognize ‘good’ from ‘bad’, ‘good’ from ‘not good’, so there’s a huge implication here for a variety of industrial uses,” says Kar.

Machines typically recognize color by breaking it down, using conventional RGB (red, green, blue) filters, into its constituent components, then use this information to guess and essentially reproduce the original color. When you point a digital camera at a colored object and take a picture, the light from that object passes through an array of detectors with filters in front of them that differentiate the light into those primary RGB colors.

You can think of these color filters as funnels that funnel visual information or data into separate boxes, which then assign “artificial numbers to natural colors,” Kar says.

“So if you just break it down into three components [red, green, blue]there are certain limitations,” says Kar.

Instead of using filters, Kar and his team used “transmissive windows” made of this unique two-dimensional material.

“We make a machine recognize color in a very different way,” Kar says. “Instead of breaking it down into its main red, green and blue components, when colored light appears, for example, on a detector, instead of just looking for these components, we use the entire spectral information. And in addition from that, we use techniques to modify and encode them, and store them in different ways, so it provides us with a set of numbers that helps us recognize the original color in a much more unique way than the conventional method.”

When light passes through these windows, the machine treats the color as data; Built-in are machine learning models that look for patterns to better identify matching colors analyzed by the device, says Ostadabbas.

“A-Eye can continually improve color estimation by adding all corrected guesses to its training database,” the researchers wrote.

Davoud Hejazi, Ph.D. in Northeast Physics. student, contributed to the research.

For media inquiriesplease contact [email protected].