Researchers from the California Institute of Technology (Caltech) discovered that a deep-learning technology tag, known as Neural-Fly, could assist flying robots known as “drones” in adapting to any weather conditions.

Drones are now flown under controlled conditions, without wind, or by people using software or remote controls. The flying robots have been trained to take off in formation in the open air, although these flights are typically undertaken under perfect conditions.

However, for drones to autonomously perform important but mundane duties, such as package delivery or airlifting injured drivers from traffic accidents, they must be able to adapt to real-time wind conditions.

With this, a team of Caltech engineers has created Neural-Fly, a deep-learning technology that enables drones to adapt to new and unexpected wind conditions in real-time by merely adjusting a few essential parameters. Neural-Fly is discussed in newly published research titled “Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds” in Science Robotics.

The issue is that the direct and specific effect of various wind conditions on aircraft dynamics, performance, and stability cannot be accurately characterised as a simple mathematical model.

– Soon-Jo Chung, Bren Professor of Aerospace and Control and Dynamical Systems and Jet Propulsion Laboratory Research Scientist

Chung added that they employ a combined approach of deep learning and adaptive control that enables the aircraft to learn from past experiences and adapt to new conditions on the fly, with stability and robustness guarantees, as opposed to attempting to qualify and quantify each effect of the turbulent and unpredictable wind conditions they frequently encounter when flying.

Neural-Fly was evaluated at Caltech’s Center for Autonomous Systems and Technologies (CAST) utilising its Real Weather Wind Tunnel, a 10-foot-by-10-foot array of more than 1,200 tiny computer-controlled fans that enables engineers to mimic everything from a mild breeze to a gale.

Numerous models derived from fluid mechanics are available to researchers but getting the appropriate model quality and tweaking that model for each vehicle, wind condition, and operating mode is difficult.

Existing machine learning methods, on the other hand, demand massive amounts of data for training, but cannot match the flying performance attained by classical physics-based methods. Adapting a complete deep neural network in real-time is a monumental, if not impossible, undertaking.

According to the researchers, Neural-Fly addresses these challenges by utilising a technique known as separation, which requires only a few parameters of the neural network to be altered in real-time. This is accomplished using their innovative meta-learning technique, which pre-trains the neural network so that only these critical parameters need to be changed in order to successfully capture the changing environment.

After only 12 minutes of flying data, autonomous quadrotor drones outfitted with Neural-Fly learn how to respond to severe winds so well that their performance improves dramatically as judged by their ability to precisely follow a flight route.

When compared to drones equipped with current state-of-the-art adaptive control algorithms that identify and respond to aerodynamic effects but lack deep neural networks, the error rate following that flight path is between 2.5 to 4 times lower.

Landing may appear more difficult than flight, however, Neural-Fly can learn in real-time, unlike previous systems. As a result, it can react on the fly to wind variations and does not require post-processing.

In-flight tests were done outside of the CAST facility; Neural-Fly functioned just as well as it did in the wind tunnel. Additionally, the researchers showed that flight data collected by one drone can be transferred to another, establishing a knowledge pool for autonomous cars.

The drones were outfitted with a typical, off-the-shelf flight control computer utilised by the drone research and enthusiast communities. Neural-Fly was built into an onboard Raspberry Pi 4 computer, which is the size of a credit card and costs roughly $20.