Periodic Reporting for period 3 - TraceBot (TRACEABLE ROBOTIC HANDLING OF STERILE MEDICAL PRODUCTS)
Okres sprawozdawczy: 2023-10-01 do 2025-03-31
Creating new medical, pharmaceutical, or health care products is always based on testing a huge number of samples. The need for microbial safety is inherent to life science suppliers and every pharmaceutical manufacturing process. Sterility testing is thus a critical and very common process in health care domain. These sample sterility testing processes are still done manually, by human operators, despite the contamination risk, the numerous tests required every day, and the repeatability of the operation. Health-care laboratories are relying on human labour, because robots cannot yet be trusted to perform life-critical tasks, since they require a flexibility unavailable from high throughput automation and are therefore still carried out manually, with tedious quality control & sign-off steps needed by regulations. TraceBot aims at bringing agile and flexible robotics to the pharmaceutical domain, by addressing the main technical barriers that are keeping robots out of the lab: the traceability and verification capabilities. TraceBot will combine Digital Twin Technology and multi-modal verification processes to provide traceable manipulation actions. These checking actions will permit the robotic system to reason on its current state, detect any deviation to be counterbalanced, and create the audit trial required to comply with medical regulations. Advanced 3D perception solutions and well-designed dexterous grippers will permit TraceBot to handle complex items. Finally, the usage of easy programming concepts, such as skill programming, teaching by demonstration and intuitive interfaces, will permit the operator to easily adapt the process to novel products or tasks.
We have been focusing on setting the foundations of the conceptual framework of TraceBot. The use-case selected has been thoroughly studied. We brought the analysis from a human perspective towards a robotic point of view, identifying the technical needs as well as the challenges that are inherent to the execution of such robotic process. Furthermore, we defined the generic verification process required to validate the successive demonstrators developed.
Preliminary designs and experimentations have been realized for the artificial skin patches that will be inserted into the dexterous gripper. This tactile layer will permit to analyse, improve and verify the good interaction of the gripper with the manipulated objects. A Skill framework system has been implemented.
The encapsulation of any robot behaviour within skills enables to harmonize the component structure and ease the collaborative development in the group. It also provides higher-level programming blocks which can be more easily assembled by a non-roboticist user through an interface which proposes relevant skills to combine based on their impact on the environment.
Based on control interfaces provided in ROS, we implemented a generic task or constraint model, in which specific and complex control laws are generated at run-time based on configuration files and a set of standard task schemes which can be stacked together.
We implemented a first structure for performing a systematic tracing of the operations conducted with the robot. This traceability is now embedded in the core of the skill framework so that all components are based on traceable skills. We implemented first examples of verification items, based on perception, and started the design of ontology-based verification, in which the state of the environment is validated at a semantic level to confirm the good execution of an action, and confirm the possibility to execute the next one in the plan. The verification and traceability strongly rely on the development of the Digital Twin, which combines a digital and dynamic representation of the robot and its environment with knowledge representation and reasoning layers. The framework of management of this reasoning solution has been defined, together with the conceptual layers associated. We started to specialize the simulation for the TraceBot use-case, representing both the robot and its environment.
A first software architecture has been defined, and first mockups of functionalities have been assembled, thanks to the interfacing modalities provided by the skill framework. The simulation, employing dynamic emulation with Gazebo, illustrates the usage of the skill framework, the control of the dual arm system, the connection with the digital twin, while providing a tracing of the operations conducted.
Finally, to raise awareness of the project, we put in place several means to disseminate our actions, through Advisory Board meetings, the website and our monthly dashboard, LinkedIn posts and YouTube videos.
Preliminary designs and experimentations have been realized for the artificial skin patches that will be inserted into the dexterous gripper. This tactile layer will permit to analyse, improve and verify the good interaction of the gripper with the manipulated objects. A Skill framework system has been implemented.
The encapsulation of any robot behaviour within skills enables to harmonize the component structure and ease the collaborative development in the group. It also provides higher-level programming blocks which can be more easily assembled by a non-roboticist user through an interface which proposes relevant skills to combine based on their impact on the environment.
Based on control interfaces provided in ROS, we implemented a generic task or constraint model, in which specific and complex control laws are generated at run-time based on configuration files and a set of standard task schemes which can be stacked together.
We implemented a first structure for performing a systematic tracing of the operations conducted with the robot. This traceability is now embedded in the core of the skill framework so that all components are based on traceable skills. We implemented first examples of verification items, based on perception, and started the design of ontology-based verification, in which the state of the environment is validated at a semantic level to confirm the good execution of an action, and confirm the possibility to execute the next one in the plan. The verification and traceability strongly rely on the development of the Digital Twin, which combines a digital and dynamic representation of the robot and its environment with knowledge representation and reasoning layers. The framework of management of this reasoning solution has been defined, together with the conceptual layers associated. We started to specialize the simulation for the TraceBot use-case, representing both the robot and its environment.
A first software architecture has been defined, and first mockups of functionalities have been assembled, thanks to the interfacing modalities provided by the skill framework. The simulation, employing dynamic emulation with Gazebo, illustrates the usage of the skill framework, the control of the dual arm system, the connection with the digital twin, while providing a tracing of the operations conducted.
Finally, to raise awareness of the project, we put in place several means to disseminate our actions, through Advisory Board meetings, the website and our monthly dashboard, LinkedIn posts and YouTube videos.
The originality of our skill framework is that it relies on a simple configuration file, or process, which can be either edited manually by a robot-aware programmer, or be the outcome of any advanced graphical programming interface. A relevant feature provided by our skill framework is the compatibility with standard ROS components. This way, we reduce the learning curve to use the skill framework, and thus permit a seamless collaboration and integration.
At a lower level, the advantage of our constraint-based robot controller is that the programmer focuses on the description of the tasks, while the generation of the appropriate robot joint motion is automatically deduced at run-time. Such approach permits to simplify control law implementation, and to promote task model reuse in new processes.
Additionally, we have developed an innovative comprehensive task analysis methodology that can provide guidelines for the design of multi-fingered robotic grippers. The proposed methodology combines a human centred gesture analysis and an object centred grasp stability analysis. On the contrary to state-of-the-art works, the video analysis is employed in our methodology to identify and classify the grasp types involved in the whole process, as well as determining the frequency of use of each elementary surface of the hand when handling the use-case objects.
We have also developed some preliminary designs of multi-layered sandwich tactile sensing patches. The objective is to equip modular robotic fingers with a thin perception layer able to give touch information between the fingers and the various objects to be manipulated.
Furthermore we have successfully implemented a first version of a 'digital twin', which is a digital replica of the robot of the TraceBot use case. It will allow the robot to better understand its own actions, by imaging how the world around him should look like and behave, when doing sterility testing. The digital twin consists of two major building blocks: 1) a game engine-based simulation component that allows the robot to simulate actions and their outcomes in a sophisticated virtual world and 2) a knowledge representation that contains common knowledge about the sterility testing use case, the robot itself and for example objects that the robot has to handle. The work conducted is progressing the state of the art in robotics by creating systems that do not blindly execute a simple sequence of actions without knowing what they are doing. We are developing an approach that will enable robots to reason about their actions in a sophisticated way by the aforementioned modalities, and therefore support the development of future robots that are more competent in handling complex manipulation tasks.
At a lower level, the advantage of our constraint-based robot controller is that the programmer focuses on the description of the tasks, while the generation of the appropriate robot joint motion is automatically deduced at run-time. Such approach permits to simplify control law implementation, and to promote task model reuse in new processes.
Additionally, we have developed an innovative comprehensive task analysis methodology that can provide guidelines for the design of multi-fingered robotic grippers. The proposed methodology combines a human centred gesture analysis and an object centred grasp stability analysis. On the contrary to state-of-the-art works, the video analysis is employed in our methodology to identify and classify the grasp types involved in the whole process, as well as determining the frequency of use of each elementary surface of the hand when handling the use-case objects.
We have also developed some preliminary designs of multi-layered sandwich tactile sensing patches. The objective is to equip modular robotic fingers with a thin perception layer able to give touch information between the fingers and the various objects to be manipulated.
Furthermore we have successfully implemented a first version of a 'digital twin', which is a digital replica of the robot of the TraceBot use case. It will allow the robot to better understand its own actions, by imaging how the world around him should look like and behave, when doing sterility testing. The digital twin consists of two major building blocks: 1) a game engine-based simulation component that allows the robot to simulate actions and their outcomes in a sophisticated virtual world and 2) a knowledge representation that contains common knowledge about the sterility testing use case, the robot itself and for example objects that the robot has to handle. The work conducted is progressing the state of the art in robotics by creating systems that do not blindly execute a simple sequence of actions without knowing what they are doing. We are developing an approach that will enable robots to reason about their actions in a sophisticated way by the aforementioned modalities, and therefore support the development of future robots that are more competent in handling complex manipulation tasks.