AI-based CCAM: Trustworthy, Explainable, and Accountable

Project Information

AIthena

Grant agreement ID: 101076754

DOI

10.3030/101076754

Project closed

EC signature date 27 July 2022

Start date 1 November 2022

End date 31 October 2025

Funded under

• Climate, Energy and Mobility

Total cost

€ 5 999 547,50

EU contribution

€ 5 999 547,50

Coordinated by

FUNDACION CENTRO DE TECNOLOGIAS DE INTERACCION VISUAL Y COMUNICACIONES VICOMTECH
Spain

Periodic Reporting for period 1 - AIthena (AI-based CCAM: Trustworthy, Explainable, and Accountable)

Reporting period: 2022-11-01 to 2024-04-30

Summary of the context and overall objectives of the project

The increasing use of AI in vehicles and transport requires humans to trust AI decisions blindly. To gain social acceptance, developing trustworthy AI is essential. This involves balancing robustness, privacy, explainability, accountability, and ethics.

AITHENA proposes a harmonized, human-centric methodology for AI-based CCAM solutions, focusing on perception, situational awareness, decision-making, and traffic management. This methodology emphasizes trustworthy AI pillars—accuracy, explainability, accountability, privacy, and ethics—to serve diverse end users, including drivers, function developers, and certification/legal bodies.

AITHENA advances three AI aspects: data management, AI model development, and testing and validation approaches. It focuses on developing explainable AI (XAI) concepts, such as physically informed neural networks, deep hybrid learning, and reinforcement learning with explainable layers. Additionally, it ensures scalable, transparent, and unbiased XAI training processes through data generation, processing, and traceability.

The RTD approach demonstrates use cases across CCAM layers: perception, situational awareness, decision-making, and mobility. These use cases propose extensions to existing standards, testing, and certification approaches.

Finally, AITHENA provides datasets and tools for future XAI system development on a cloud platform that complies with the European Dataspaces approach and GAIA-X4Future Mobility architecture. This adherence to European values of data protection, authenticity, and trust enhances European visibility and impact in the digital ecosystem.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

In this first period of the project, the AITHENA consortium has focused on the creation of the “AITHENA methodology for explainable, trustworthy and human centric AI-based system and function development”. It is a document that forms the basis for all other AITHENA activities, as it contains guidance and tools for developers and tested of CCAM functions, as well as regulators, integrators and other stakeholders.
Main technical activities have focused on gathering representative data to be able to train/test/validate the AI models. Recording campaigns were carried out by different partners, creating multi sensor data (camera, LiDAR and radar) from roads and test tracks.
Other critical situations, like crashes of traffic jams were generated in simulated environments. Along with the generated data, AITHENA proposes the concept of Data Cards of the datasets as a summary of datasets content as well as information on ethics, privacy, inclusion of sensitive data or representativeness.
An anonymization tool is also under construction in AITHENA to follow GDPR, where the tool either blurs or uses Generative AI to substitute faces/bodies of people in the recorded scenes and license plates of the vehicles.
The activities on AI algorithm development focused first on the definition of the MLOps Card, a template to design MLOPs frameworks devised to fulfil the AITHENA methodology, that can be used during the development and design of the AI models. Several implementations of the template have been produced by partners to work on the development of AI models under the different AITHENA use cases (UC).
As a result, several AI models were developed (designed, trained, evaluated) in the project for UC1-3: Perception, Situational Awareness, and Decision Making, making use of multi-sensor data, and diverse scenarios and situations. These models include saliency maps and other types of visualization, so that developers or other users can get feedback from the decision making models on why these decisions are being made and see the information that the algorithms are taking into account. Other models were developed to test edge cases, like extreme weather conditions or safety critical situations like overtaking of a cyclist. These models were trained to detect the weather conditions and to classify the situations as risky/not risky to alert the driver and make the driving experience more safe.
In the context of UC4 Traffic Management, simulations were designed and executed to measure the impact and behaviour of mixed traffic environments, with various percentages of presence of AI-based autonomous vehicles (AV), and existence of a Traffic Management (TM) system that sends commands to connected AVs to solve critical situations, and improve traffic.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

AITHENA has produced several scientific papers (listed below) in this first period of the project aligned the main objective of enhancing intrinsic and extrinsic trust at the different features of the trusted AI paradigm. In this period AITHENA has advance in
- Creation of report templates on explained behaviour of neural networks for common tasks (detection, classification, recognition).
- Creation of ontology with semantic terms described in detail (description, examples, relations to other concepts) to enhance explainability capacities
- Creation of metadata template that explains the type of information contained at intermediate ML stages and at all steps of the data management cycle.
- New approaches for covering physically informed neural networks
- Sensor fusion for enhanced robustness against conflicting perception.
- Utilisation of novel cross-dataset evaluation mechanisms to meta-evaluate accuracy metrics
This advances are reflected in the following scientific papers and in the public deliverables of the project.
• “Explainable Multi-Camera 3D Object Detection with Transformer-Based Saliency Maps”: Presented at the Machine Learning for Autonomous Driving Symposium (NeurIP S 2023) in New Orleans, USA.
• “MultiCorrupt: A Multi-Modal Robustness Dataset and Benchmark of LiDAR-Cam era Fusion for 3D Object Detection”: Accepted at the IEEE Intelligent Vehicles Symposium (IV).
• “V2AIX: A Multi-Modal Real-World Dataset of ETSI ITS V2X Messages in Public Road Traffic”: Submitted to the IEEE Intelligent Transportation Systems Conference (ITSC).
• “Digital twin for synthetic data generation - application for automated driving systems” by Hassan Hotait and Alexandru Forrai.
• “Runtime Safety Assurance of Autonomous Vehicles” by A. Forrai, V. Neelgundmath, K.K. Unni, and I. Barosan. Trustworthiness Assurance Assessment A journal article titled "Trustworthiness Assurance Assessment for High-Risk AI-Based Systems" has been published, addressing critical aspects of AI trustworthiness assurance

AITHENA has also developed the first version of a guide and estate of the art report on Human centric AI published as deliverable “D1.1 AITHENA Methodology for explainable, trustworthy and human centric AI-based system and function development”. The goal of the document is to give developers, auditors and the general public a guide with references to different ethical aspect of AI and how they are solved.

AITHENA XAI for CCAM

AITHENA approach, technologies and use cases

AITHENA annonimization