Lightweight AI detects marine litter from space
The number of satellites in orbit is growing faster than the bandwidth available to download this data. To remedy this, Edge SpAIce(opens in new window) set out to develop and demonstrate artificial intelligence (AI) capable of analysing Earth observation (EO) imagery on board a satellite, limiting downloads to the most valuable data. The team chose the identification of marine litter as a test case. But according to project coordinator Pauline Audenino, senior project director at Agenium Space(opens in new window) in France, the underlying technology has much broader applications. In the aftermath of an emergency, for example, useful information can be identified while the satellite is still in orbit, rather than waiting for imagery to be sent to Earth for analysis: “A satellite goes over a disaster area and promptly provides images of changes – what has been destroyed, or not.” The project brought together four partners with very different expertise. The National Technical University of Athens(opens in new window) (NTUA) developed and trained the AI models, while project coordinator Agenium Space worked on making these models small enough to run on board a satellite. CERN(opens in new window) provided the open-source technology needed to adapt the models to specialist computing hardware, and EnduroSat(opens in new window) supplied satellite imagery and led the in-orbit validation.
Shrinking AI for space constraints
NTUA developed a deep neural network (DNN) using labelled satellite imagery. The model was trained to recognise marine litter, distinguishing this from the many other features seen in ocean imagery. The team at Agenium Space managed to reduce this neural network (containing more than 9 million parameters) to 1 % of its original size. Despite the dramatic reduction, the lightweight model could still accurately identify marine litter while avoiding false positives. With the complete on-board detection package reduced to around one megabyte in size, it was uploaded and run on the BALKAN 1 for a successful end-to-end demonstration of image acquisition followed immediately by on-board processing.
Building European space capabilities
“The other objective, only on ground, was to use a sovereign European board,” says Audenino. NanoXplore is currently Europe’s only manufacturer of space-grade field-programmable gate array (FPGAs) – reconfigurable hardware chips that form the backbone of satellite computers. But running AI on this hardware also requires suitable software. Edge SpAIce extended CERN’s open-source HLS4ML framework(opens in new window) to support the technology, successfully running a small neural network on NanoXplore’s NG-Ultra chip in ground tests. Aside from technical results, the consortium has shared its work through publications and international conferences, while select partners have submitted applications for a new Horizon Europe project, ALMAGEST, to further develop Edge SpAIce’s work. For now, the project has shown that DNNs don’t have to remain on the ground. Giving satellites more autonomy to interpret what they see could help EO systems respond faster to crises, delivering tangible benefits on the ground.
