ReliablE in-Vehicle pErception and decisioN-making in complex environmenTal conditionS

Projektinformationen

EVENTS

ID Finanzhilfevereinbarung: 101069614

DOI

10.3030/101069614

Projekt abgeschlossen

EK-Unterschriftsdatum 3 Juni 2022

Startdatum 1 September 2022

Enddatum 28 Februar 2026

Finanziert unter

• Climate, Energy and Mobility

Gesamtkosten

€ 6 812 764,66

EU-Beitrag

€ 5 534 448,00

Koordiniert durch

EREVNITIKO PANEPISTIMIAKO INSTITOUTO SYSTIMATON EPIKOINONION KAI YPOLOGISTON
Greece

Periodic Reporting for period 2 - EVENTS (ReliablE in-Vehicle pErception and decisioN-making in complex environmenTal conditionS)

Berichtszeitraum: 2024-03-01 bis 2026-02-28

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

In our everyday life as drivers, we are facing unexpected situations we need to handle in a safe and efficient way. The same is valid for Connected and Automated Vehicles (CAVs), which also need to handle these situations, to a certain extent, depending on their automation level. The higher the automation level is, the higher the expectations for the system to cope with these situations are. In the context of this project, these unexpected situations where the normal operation of the CAV is close to be disrupted (e.g. ODD limit is reached due to traffic changes, harsh weather/light conditions, imperfect data, sensor/communication failures, etc.), are called “events”. EVENTS is also the acronym of this project.
Within our scope, and, to cover a wide area of scenarios, these kinds of events are clustered under three main use cases: a) Interaction with VRUs and other vehicles, b) Non-Standard and Unstructured Road Conditions and c) Low Visibility and Adverse Weather Conditions.
EVENTS aims to create a robust and self-resilient perception and decision-making system for AVs to manage different kind of “events” on the horizon. These events result in reaching the AV ODD limitations due to the dynamic changing road environment (VRUs, obstacles) and/or due to imperfect data. The AV should continue and operate safely no matter what. When the system cannot handle the situation, an improved minimum risk manoeuvre should be put in place.
The overall objectives of EVENTS can be summarized as follows:
1. Design and implement on-board perception algorithms needed for safe driving of CAVs in complex environmental conditions.
2. Design and implement decision-making algorithms able to cope with a variety of traffic scenarios including non-standard traffic conditions.
3. Develop solutions for continuous perception system self-assessment for safe and resilient operation.
4. Integrate, test and demonstrate the developed perception and decision-making algorithms in both prototype vehicles (real conditions) and simulation environments.
5. Assess the impact of EVENTS developments and determine cost-efficient sensor suites for CAVs.
6. Disseminate and communicate project findings, increase cooperation with international stakeholders and promote project results to standardisation bodies.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

EVENTS successfully completed all planned activities, delivering a robust, self-resilient perception and decision-making system for CAVs validated across eight experiments, nine prototype vehicles and multiple testing environments.

WP3:
A comprehensive multi-modal perception stack (camera, LiDAR, radar, GNSS/INS, HD maps, V2X) was developed and validated. Key achievements include:
• Unsupervised 3D Object Detection (UNION, EXP1): TUD developed an unsupervised LiDAR/camera detection framework achieving AP=39.5 on nuScenes (more than tripling prior unsupervised baselines), without manual labels, using DINOv2 appearance clustering.
• Cooperative Perception (EXP2): A CPM fusion module improved FoV coverage from ~24% (ego-only) to over 85% with full agent cooperation and boosted night-condition recall from 0.65 to 0.94. Validated via a hybrid ViL/CARLA setup with a purpose-built digital twin.
• Radar-Based Debris Detection (EXP6): APTIV’s radar algorithm detected non-driveable objects at up to 112 m, maintaining classification F1 > 0.81 under rain. SOTIF KPI remained below 13% across all tested weather conditions.
• Advanced Radar Perception (EXP8): Perciv.AI’s ML radar pipeline achieved 1.7× higher detection rate, 4.1× better classification accuracy and 5× fewer false positives vs. Tier-1 baselines. In live adverse weather tests, the system successfully tracked an occluded cyclist in fog and rain where LiDAR failed entirely.
• Traffic Sign Detection & HD-Map Update (EXP4): Hitachi’s synthetic data generation framework produced traffic sign detection models matching real-data performance, with lane boundary estimation enabling safe path generation through roadworks (~30 cm lateral error, detected at 3s/50 km/h).
• Perception Self-Assessment (EXP3, EXP5, EXP7): SA mechanisms achieved 100% disturbance detection recall (avg. delay<2.2 s, UULM), AUROC up to 0.9580 for LiDAR integrity monitoring via activation injection (WMG/nuScenes) and F1=0.70 for camera-LiDAR consistency in highway merging (EXP5).

WP4:
• Platooning and Cooperative Decision-Making (EXP2): An end-to-end platooning system validated via ViL achieved 70% success rate in roundabout scenarios. Late-fusion cooperative perception reduced b-minADE to 1.40 m vs. 1.745 m baseline.
• Path Planning for Roadworks (EXP4): A Bézier spline trajectory generator validated on a Renault Twizy achieved mean lateral error of 0.016 m, suitable for urban/roadworks scenarios up to 45 km/h.
• Collision Avoidance (EXP8): An MPPI controller validated on a Toyota Prius at temperatures of −5°C successfully avoided a dynamically occluded cyclist under snow/ice conditions where the Autoware baseline failed.
• Joint Prediction and Planning (EXP7): ICCS’s joint model outperformed the modular baseline in NuPlan simulations (ADE: 11.5 m vs. 19.2 m), with a collision rate of 3.1% in static evaluation.
• Motion Prediction (EXP2 & EXP5): The HiVT-64 map-less model trained on Argoverse 1 achieved overall minADE of 1.29 m on real highway merging data, demonstrating robust generalization to operational domain conditions.

WP5:
All modules were fully integrated into nine prototype vehicles, validated via ViL/SciL (18-container Docker stack for EXP2) and real-world tests, with SOTIF compliance testing across all eight experiments (D5.3).

WP6:
A V-model/SOTIF evaluation methodology, combining component benchmarking and system-level acceptance criteria, was applied across all eight experiments using public benchmarks (nuScenes, KITTI, Argoverse 1) and newly collected datasets (M40 motorway, roadworks, debris, adverse weather); full results in D6.2.

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

EVENTS produced technically significant results advancing the state of the art across perception and decision-making for CCAVs:
• Unsupervised 3D perception (UNION, NeurIPS 2024): AP=39.5 on nuScenes (more than tripling prior unsupervised baselines), with no manual labels, reducing dependence on costly annotation.
• Cooperative perception: Multi-agent cooperation improved night-condition recall from 0.65 to 0.94; localization uncertainty (not sensor quality) identified as binding bottleneck above ~1.5 m error; novel design guidance for V2X systems.
• AI radar vs. LiDAR in adverse weather: 4.1× better classification and 5× fewer false positives over Tier-1 baselines; successfully tracked occluded cyclists in fog/rain where LiDAR failed entirely, establishing AI radar as a viable adverse-weather sensor modality.
• Perception self-assessment without ground truth: Activation injection achieved AUROC=0.9580 (nuScenes), previously unreported for this class of integrity monitoring, with 100% disturbance recall at <2.2s delay in real-world EXP3 tests.
• Scalable synthetic training for traffic signs: Hitachi’s patch-augmentation approach produces near-parity training data for non-standard and country-specific signs, directly addressing the long-tail annotation problem.
• Collision avoidance at the friction limit: MPPI with combined braking/steering prevented collisions under snow/ice (μ=0.1–0.3) where standard braking-only baselines failed, a demonstrated safety advance for high-latitude AV operation.

Potential Impacts and Key Needs for Further Uptake
• Standardization and regulation: EVENTS results have been fed into ISO, SAE, ASAM, UNECE and BSI working groups. Continued engagement is needed to establish requirements for perception self-assessment and adverse weather ODD definitions; the SOTIF methodology (D5.3) is available as an industry reference.
• Further R&D: SA mechanisms need domain-specific fine-tuning and temporal learning; the radar pipeline requires broader adverse weather validation; joint prediction and planning should be extended to denser traffic and higher automation levels.
• Demonstration infrastructure and data availability: The hybrid ViL framework (EXP2) offers a reusable platform for multi-agent validation. Open release of the road debris (EXP6), UK motorway SA (EXP7) and adverse weather radar (EXP8) datasets is a priority, as these fill gaps in existing public benchmarks.
• Commercialization: Perciv.AI’s growth from 3 to 33 employees illustrates the commercial traction of AI radar. Cost-benefit analysis (D6.3) favours radar-centric sensor suites for adverse weather, relevant to OEM and Tier-1 decision-making. Bridging the current TRL 6–7 gap to production-grade maturity is the key next step.
• IPR and technology transfer: Exploitable results spanning algorithms, datasets and system methodologies have been identified. Clear licensing frameworks and targeted OEM/Tier-1 engagement are essential for the self-assessment, cooperative perception and radar modules.