TRACEABLE ROBOTIC HANDLING OF STERILE MEDICAL PRODUCTS

Project Information

TraceBot

Grant agreement ID: 101017089

DOI

10.3030/101017089

Project closed

EC signature date 16 December 2020

Start date 1 January 2021

End date 31 March 2025

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 6 999 905,00

EU contribution

€ 6 999 905,00

Coordinated by

BIOLAGO EV
Germany

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Deliverables

Documents, reports (33)

Verification and validation plan 1

D1.4Verification and validation plan

ROS middle-ware mock-up, demonstrating UR arms control using ROS simulated environment

D6.1 - ROS middle-ware mock-up, demonstrating UR arms control using ROS simulated environment

Impact dashboard 1

twice a yearD7.1 Impact dashboard (monthly, concise single page summary) PUBLIC

Skill-based framework description

D3.1 Skill-based framework description. Description of the modelling of robotic skills developed in the extension of classical models

Knowledge management checklist

D8.5 Knowledge management checklist and asset register - ongoing

Impact dashboard 4

Twice a year

Traceability framework for laboratory automation:

D4.1 Traceability framework for laboratory automation: fully functional first version including first verification examples in one of the modalities

Construction of mechanical framework to mount robotics arms within a laboratory hood.

D6.3 - Construction of mechanical framework to mount robotics arms within a laboratory hood. Integration of setup using new components with the robotic arm. Includes acceptance testing of ROS middle-ware to ensure services provide full control of robotic arm along with integrated components.

Design guidebook

D1.3Design guidebook

Impact dashboard 3

Twice a year

Regulatory engagement plan

D7.5 Regulatory engagement plan (M12)

Integrated Traceability framework using tactile, visual and functional task verification

D4.3 Integrated Traceability framework using tactile, visual and functional task verification: description, evaluation and open source software

Technical specifications as recommendations for the design of the multi-fingered gripper

D2.1 – Technical specifications as recommendations for the design of the multi-fingered gripper

Communication plan

D7.2 Communication plan and materials

Framework: Definition of the conceptual and reasoning framework and semantic models

D5.1 Framework: Definition of the conceptual and reasoning framework and semantic models. Fundamental simulation aspects are available in the TST.

Use Case Specification 2

D1.1Use case Specification

Report or Publication about lab process action learning by analysing human demonstrations in VR

D5.5 Report (publication) on laboratory process action learning by analysing human demonstrations in VR

Intuitive programming interface Framework

D3.3: Intuitive programming interface Framework: Description of the operation programming interface including task demonstration mechanism

Control strategies of hand mounted on a robotic arm performing unitary operations

D2.3 Control strategies of hand mounted on a robotic arm performing unitary operations

Impact dashboard 2

Twice a year

Initial tactile, visual and functional task verification

D4.2 Initial tactile, visual and functional task verification: description, evaluation and open source software

Initial setup using 2 x Robotic arms with prototypes or similar functional components attached, using ROS middle-ware mock-up

D6.2 - Initial setup using 2 x Robotic arms with prototypes or similar functional components attached, using ROS middle-ware mock-up. The integration will not be mounted within a framework at this point and will not be within a sterile environment

Project Handbook, communications infrastructure

D8.1 Project Handbook, communications infrastructure

Sensor-based control strategies

D3.2: Sensor-based control strategies. Description of the perception modalities and sensor-based robotic controllers developed to handle the main unimanual and bimanual manipulations

Software models (final)

D5.3 Software models (Final): Final definition of the software interfaces and software creating the Audit Trail. Related to Milestone 3.

Use case Specification 1

D1.1Use case Specification

Rapid set-up of laboratory automation cell

D6.5 - Rapid set-up of laboratory automation cell. Operations and Maintenance Manuals. Full release of Git repository with ROS framework and packages. Final release of mechanical and electrical schematics. Compilation of guidelines towards certification (O3)

Software models (basis)

D5.2 Software models (Basis): Software interfaces for first demonstrator defined and required simulation aspects are available. Related to Milestone 2.

Verification and validation plan 2

D1.4Verification and validation plan

Dissemination plan

D7.3 Dissemination plan and activities; outward-facing website

System integrated readily to be placed within a sterile enclosure to accommodate all of the hardware required to automate the sterile testing

D6.4 - System integrated readily to be placed within a sterile enclosure to accommodate all of the hardware required to automate the sterile testing (O6)

System Architecture

D1.2System Architecture

Report or Publication about robotic minds eye reasoning for verification in laboratory environments

D5.4 Report (publication) about robotic minds eye reasoning for verification in lab environments

Open Research Data Pilot (4)

Data Management Plan 2

Data Management Plan update

Data Management Plan 1

D8.2 Data Management Plan

Data Management Plan 4

Data Management Plan update

Data Management Plan 3

Data Management Plan update

Publications

Book chapters (2)

Robotizing the Sterility Testing Process: Scientific Challenges for Bringing Agile Robots into the Laboratory

Author(s): Anthony Remazeilles, Irati Rasines, Asier Fernandez, Maike Neumann, Michael Beetz, Mathieu Grossard, Clive Hellingman, Carl-Helmut Coulon, Torben Cichon, Florian Gosselin, Markus Vincze
Published in: Lecture Notes in Networks and Systems, ROBOT2022: Fifth Iberian Robotics Conference, 2022, Page(s) 223-234
Publisher: Springer International Publishing
DOI: 10.1007/978-3-031-21065-5_19

Challenges of Depth Estimation for Transparent Objects

Author(s): Jean-Baptiste Weibel, Paolo Sebeto, Stefan Thalhammer, Markus Vincze
Published in: Lecture Notes in Computer Science, Advances in Visual Computing, 2023, Page(s) 277-288
Publisher: Springer Nature Switzerland
DOI: 10.1007/978-3-031-47969-4_22

Conference proceedings (16)

Measuring the Sim2Real Gap in 3D Object Classification for Different 3D Data Representation

Author(s): J. Weibel, R. Rohrböck, M. Vincze
Published in: Lecture Notes in Computer Science: 13th International Conference on Computer Vision Systems, 2021
Publisher: Springer
DOI: 10.1007/978-3-030-87156-7_9

"NaivPhys4RP - towards Human-Like Robot Perception ""Physical Reasoning Based on Embodied Probabilistic Simulation"""

Author(s): F. K. Kenfack, M. Neumann, P. Mania, T. Tan, F. A. Siddiky, R. Weller, G. Zachmann, M. Beetz
Published in: 2022
Publisher: IEEE
DOI: 10.1109/humanoids53995.2022.10000153

An open and flexible robot perception framework for mobile manipulation tasks

Author(s): P. Mania, S. Stelter, G. Kazhoyan, M. Beetz
Published in: 2024
Publisher: IEEE

A Modular Framework for Knowledge-Based Servoing: Plugging Symbolic Theories into Robotic Controllers

Author(s): M. Huerkamp, K. Dhanabalachandran, M. Pomarlan, S. Stelter, M. Beetz
Published in: International Conference on Agents and Artificial Intelligence (ICAART), 2025
Publisher: SCITEPRESS

ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformers

Author(s): Philipp Ausserlechner, David Haberger, Stefan Thalhammer, Jean-Baptiste Weibel, Markus Vincze
Published in: 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, Page(s) 463-469
Publisher: IEEE
DOI: 10.1109/icra57147.2024.10611464

Towards Autonomous Verification: Integrating Cognitive AI and Semantic Digital Twins in Medical Robotics

Author(s): P. Mania, F. K. Kenfack, M. Neumann, M. Beetz
Published in: IEEE ICRA 2025, 2025
Publisher: IEEEE

Perception through Cognitive Emulation - A Second Iteration of NaivPhys4RP for Learningless and Safe Object Recognition and Pose Estimation of (Transparent) Objects from Poor Sensor Data

Author(s): F. K. Kenfack, M. Neumann, P. Mania, M. Beetz
Published in: 2024
Publisher: IEEE

Analysis of 3D shape representations in presence of corrupted data

Author(s): R. Rohrböck, J. Weibel, M. Vincze
Published in: "Austrian Robotics Workshop ""ARW""", 2021
Publisher: "Austrian Robotics Workshop ""ARW"""

SporeAgent: Reinforced Scene-level Plausibility for Object Pose Refinement

Author(s): Dominik Bauer, Timothy Patten, Markus Vincze
Published in: IEEE/CVF Winter Conference on Applications of Computer Vision, WACV, 2022
Publisher: https://openaccess.thecvf.com
DOI: 10.48550/arxiv.2201.00239

PyraPose: Feature Pyramids for Fast and Accurate Object Pose Estimation under Domain Shift

Author(s): S. Thalhammer, M. Leitner, T. Patten, M. Vincze
Published in: IEEE International Conference on Robotics and Automation (ICRA), 2021
Publisher: IEEE
DOI: 10.1109/icra48506.2021.9562108

Spectro-Temporal Recurrent Neural Network for Robotic Slip Detection with Piezoelectric Tactile Sensor

Author(s): Théo Ayral, Saifeddine Aloui, Mathieu Grossard
Published in: 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2023, Page(s) 573-578
Publisher: IEEE
DOI: 10.1109/aim46323.2023.10196263

A Methodology for early design specifications of robotic grippers

Author(s): Jonatan Martín Escorcia-Hernández, Mathieu Grossard, Florian Gosselin
Published in: 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2023, Page(s) 616-622
Publisher: IEEE
DOI: 10.1109/aim46323.2023.10196258

Enabling Classification of Heavily-occluded Objects through Class-agnostic Image Manipulation

Author(s): B. Gallauner, S. Thalhammer, M. Vincze
Published in: OAGM Workshop 2021 Computer Vision and Pattern Analysis Across Domains, 2021
Publisher: Verlag der Technischen Universität Graz

Object Placement in an Unknown Environment as Part of a Pick-and-Place Pipeline

Author(s): C. Eder, T. Patten, J. Weibel, M. Vincze
Published in: 2021
Publisher: Austrian Robotics Workshop

An iterative method for solving the inverse kinematic problem of three-joints robotic fingers with distal coupling.

Author(s): Jonatan Martín Escorcia-Hernández, Mathieu Grossard, Florian Gosselin, Clémence Dubois
Published in: 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2024, Page(s) 623-628
Publisher: IEEE
DOI: 10.1109/aim46323.2023.10196182

ReAgent: Point Cloud Registration using Imitation and Reinforcement Learning

Author(s): [1] D. Bauer, T. Patten, M. Vincze
Published in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Publisher: CVPR
DOI: 10.1109/cvpr46437.2021.01435

Peer reviewed articles (6)

Self-supervised Vision Transformers for 3D pose estimation of novel objects

Author(s): Stefan Thalhammer, Jean-Baptiste Weibel, Markus Vincze, Jose Garcia-Rodriguez
Published in: Image and Vision Computing, Issue 139, 2024, Page(s) 104816, ISSN 0262-8856
Publisher: Elsevier BV
DOI: 10.1016/j.imavis.2023.104816

Challenges for Monocular 6-D Object Pose Estimation in Robotics

Author(s): Stefan Thalhammer, Dominik Bauer, Peter Hönig, Jean-Baptiste Weibel, José García-Rodríguez, Markus Vincze
Published in: IEEE Transactions on Robotics, Issue 40, 2025, Page(s) 4065-4084, ISSN 1552-3098
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/tro.2024.3433870

ReFlow6D: Refraction-Guided Transparent Object 6D Pose Estimation via Intermediate Representation Learning

Author(s): Hrishikesh Gupta, Stefan Thalhammer, Jean-Baptiste Weibel, Alexander Haberl, Markus Vincze
Published in: IEEE Robotics and Automation Letters, Issue 9, 2024, Page(s) 9438-9445, ISSN 2377-3766
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
DOI: 10.1109/lra.2024.3455897

Robots Adapting to the Environment: A Review on the Fusion of Dynamic Movement Primitives and Artificial Potential Fields

Author(s): Irati Rasines, Itziar Cabanes, Anthony Remazeilles, Joseph Mcintyre
Published in: IEEE Access, Issue 12, 2024, Page(s) 92598-92611, ISSN 2169-3536
Publisher: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2024.3418573

Task-Oriented Methodology Combining Human Manual Gestures and Robotic Grasp Stability Analyses: Application to the Specification of Dexterous Robotic Grippers

Author(s): Jonatan Martin Escorcia-Hernandez, Mathieu Grossard, Florian Gosselin
Published in: Journal of Mechanical Design, Issue 145, 2023, ISSN 1050-0472
Publisher: American Society of Mechanical Engineers
DOI: 10.1115/1.4056493

Minimum Cost Averaging for Multivariate Time Series Using Constrained Dynamic Time Warping: A Case Study in Robotics

Author(s): Irati Rasines, Anthony Remazeilles, Miguel Prada, Itziar Cabanes
Published in: IEEE Access, Issue 11, 2023, Page(s) 80600-80612, ISSN 2169-3536
Publisher: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2023.3300720