During the 4-year duraction of the project the consortium has successfully achieved main objectives. The work performed can be summarized below:
Multiple surveys were conducted with winemakers, vineyard managers and vineyard workers in Italy and Greece, presenting the planned use-cases, in order to collect the user-requirements via questionnaires. An initial list of requirements was furnished and the system architecture was analytically defined. The planned use-cases were refined paying more focus on the inspection and harvesting of vertical trellis vineyards. Target KPI's were set.
The mobile dual-arm harvesting platform was designed and built from scratch, taking into account the different requirements from the use-cases. It consists of the main mobile platform with pseudo-omnidirectional wheels, two integrated UR10e arms and several sensors on the platform such as lidar, cameras, security system, battery manager, etc. The platform has been tested during three seasons (2021-2023) in the Gerovassileiou vineyard in Thessaloniki, performing autonomous navigation and successful harvesting experiments under challenging cases.
The inspection platform was assembled based on the Thorvals platform, carrying multiple sensors to navigate the field and assess the crops using a hyperspectral camera. The platform has done extensive field tests and data collection autonomously in Gerovassileiou vinyeard.
Multiple datasets were collected with RGBD sensors, lidars and hyperspectral cameras. New modules were created for the semantic segmentation of the vine tree, to detect the grapes and identify leaves, canes and static obstacles. Probabilistic localization algorithms were defeloped, offering reliable sensor fusion to offer the vest of different localization methods. A topological map consisting of a set of nodes and edges was also developed, defining a high-level structure of the operational environment.
Spectral libraries have been created for the non-catastrophic measurement of the sugar level of grapes, based on near-infrared sensing at different varieties.
Research for controlling the motion of the dual-arm harvesting platform was performed, towards reaching a region of interest of grapes ready to be harvested, while optimizing the view of the stem. A visual servoing algorithm was developed to reach the stem for cutting while keeping it centered on the camera’s view. The two arms are coordinating their motion throughout the operation, for achieving an optimal relative pose of the two end-effectors before harvesting. A physical unveiling dual-arm method was developed, where one arm moves the leaves out of the way for the in-hand camera to detect the stem.
To address operation planning and task scheduling for all operations in the field, the corresponding planning algorithms were developed for the operations such as, inspection, yield prediction, harvesting, etc., as well as by interconnecting the operation planner and scheduler with the system’s HMI interface for interacting with the users, i.e. the vineyard supervisor, the human workers, etc
All of the modules have been integrated in the two robot platforms. Extensive tests have been performed in the vineyard with fully autonomous inspection and harvesting. Eventually, we realized our vision where the inspection platform navigates the field autonomously, creating a crop map, estimating the yield and providing the winemakers with useful information. When the time for harvesting has come, the harvesting platform utilizes the crop map and harvest the grapes of the highest quality.
