New algorithms enable the use of adaptive autonomous underwater vehicles
Earth’s underwater ecosystems are under threat, the result of climate change, pollution and overuse, among other environmental factors. “One of our common European goals is to protect the health and biodiversity of marine environments,” says Ivan Masmitja, a researcher at the Institut de Ciències del Mar (ICM-CSIC). “However, protecting and conserving these extremely important ecosystems requires completely new, groundbreaking approaches.” One such approach is the use of adaptive autonomous underwater vehicles (AUVs), essentially underwater drones. According to Masmitja, AUVs have the potential to enable researchers to get an unparalleled look at our marine environments and collect data on a wide range of environmental factors. “The challenge is that it is very difficult to localise and track underwater objects, especially marine life,” he explains. With the support of the EU-funded AIforUTracking project, researchers at ICM-CSIC look to help solve this problem by using machine learning-based algorithms. “Moving towards the envisioned applications of marine animal tracking by autonomous vehicles, this project is at the forefront of research and directly advances the objectives outlined by the EU’s Marine Strategy Framework Directive,” adds Joan Navarro, also a researcher at ICM-CSIC who participated in this project. This research was undertaken with the support of the Marie Skłodowska-Curie Actions programme.
Reinforcement learning to track underwater targets
At the heart of the project is the use of reinforcement learning (RL). “RL is a unique corner of machine learning that aims to optimise control by looking at how an intelligent agent should act within a dynamic environment in order to achieve the desired result,” remarks Navarro. Within the framework of the project, that means using RL to help the AUV find the optimal path to track underwater targets using range-only information. But as is the case with many research projects, doing so was easier said than done. “One of the main challenges we faced was incorporating the RL algorithms into existing AUVs, many of which lacked a CPU strong enough to handle the necessary software packages,” notes Masmitja. “Instead, we had to write the RL network from scratch using basic mathematical notations.”
Machine learning helps with target localisation
With the RL written and installed, it was time to put the AIforUTracking solution to the test. For that, the project team headed to California. Placing an RL-enabled AUV into Monterey Bay, researchers successfully tracked the drone from a surface vehicle for several hours and for more than 2.5 kilometres. “For the first time ever, we demonstrated that RL algorithms can be trained and used to tackle major issues in underwater missions such as target localisation,” says Navarro.
Using algorithms to coordinate a fleet of underwater autonomous vehicles
In addition to RL, the project also worked with multi-agent reinforcement learning (MARL) algorithms. Specifically, researchers developed a new algorithm with transformers that outperformed state-of-the-art algorithms in different scenarios. Masmitja says that this represents another major milestone as MARL could be used to coordinate a fleet of vehicles to explore the ocean. “With the techniques we developed and implemented, we are making big steps toward more autonomous and adaptable vehicles for exploring, studying and monitoring the ocean and the many creatures that call it home,” he concludes. Many of the project’s results have been published in ‘Science Robotics’, one of the most prestigious journals in the robotics sector.
Keywords
AIforUTracking, biodiversity, marine environment, algorithms, autonomous underwater vehicles, marine animal tracking, climate change, drones
