Giving hardware the ability to see has enabled a variety of applications, including self-driving cars, object identification, and crop monitoring. However, unlike animal vision systems, artificial vision systems are unable to evolve in their natural environments.
“Our system could be of use in the development of unconventional applications, like panoramic motion detection and obstacle avoidance in continuously changing environments, as well as augmented and virtual reality. Currently, the size of a semiconductor optical unit, commonly used in smartphones, automobiles, and surveillance/monitoring cameras, is restricted at the laboratory level,” says Young Min Song, professor of electrical engineering and computer Science at GIST.
Dynamic visual systems that can navigate both land and water have thus yet to power the machines, prompting the researchers to develop a novel artificial vision system that closely replicates the vision of the fiddler crab, which can navigate both terrains.
Because all current systems are hemispherical, the semi-terrestrial species – fondly known as the calling crab because it looks to be beckoning with its large claws – has amphibious imaging ability and an extraordinarily broad field-of-view.
The artificial eye, which resembles a round, generally unremarkable, tiny, black ball, interprets its inputs using a combination of components that process and comprehend the light. The researchers enveloped a 3-D spherical structure in an array of flat microlenses with a graded refractive index profile and a flexible photodiode array with comb-shaped patterns. Light rays from numerous sources would always converge at the same location on the image sensor, independent of the refractive index of its surroundings, thanks to this design.
Both the amphibious and panoramic imaging capabilities were evaluated in in-air and in-water studies by imaging five objects of varying distances and directions, and the system produced constant image quality and a nearly 360° field of view in both terrestrial and aquatic situations. That is, it could see both underwater and on land when prior systems could only see in one.
When it comes to fiddler crabs, there’s more to them than meets the eye. Because they live both underwater and on land, their gigantic claws have powerful, one-of-a-kind eyesight systems. Their flat corneas, along with a graded refractive index, counteract defocusing effects caused by changes in the external environment – a significant limitation for other compound eyes.
An ellipsoidal and stalk-eye configuration gives the microscopic critters a 3-D omnidirectional field of view. To avoid attacks on large open tidal flats and to communicate and engage with partners, they’ve evolved to gaze at virtually everything at once.
A wide field-of-view (FoV) camera that reproduced the compound eyes of an insect was described in Nature in 2013, while a wide FoV camera mimicking a fisheye was reported in 2020. While these cameras can capture enormous areas at once, it is structurally difficult to get beyond 180 degrees, and commercial solutions with 360-degree FoV have lately entered the market.
With this, the crab was an excellent muse. During the testing, five adorable items (a dolphin, an aeroplane, a submarine, a fish, and a ship) were projected onto the artificial vision system from various angles. The scientists experimented with multi-laser spot imaging, and the fake images matched the simulation. They dipped the apparatus partly in water in a container to go deep.
Looking at biologically inspired light adaption strategies in the hunt for higher resolution and superior image processing approaches is a logical continuation of the work. This is an amazing feat of optical engineering and non-planar imaging, combining aspects of bio-inspired design and modern flexible electronics to attain capabilities not found in traditional cameras. The potential applications range from population surveillance to environmental monitoring.
