Detecting Document frauD and iDentity on the fly

Project Information

D4FLY

Grant agreement ID: 833704

DOI

10.3030/833704

Project closed

EC signature date 29 April 2019

Start date 1 September 2019

End date 31 August 2022

Funded under

• Secure societies - Protecting freedom and security of Europe and its citizens

Total cost

€ 6 984 727,50

EU contribution

€ 6 984 727,50

Coordinated by

VERIDOS GMBH
Germany

Periodic Reporting for period 2 - D4FLY (Detecting Document frauD and iDentity on the fly)

Reporting period: 2021-03-01 to 2022-08-31

Summary of the context and overall objectives of the project

The goal of the D4FLY project was to augment the capabilities of border authorities in countering emerging threats in document and identity verification at manual and highly automated border control points and in the issuance process of genuine documents. D4FLY examined the overall identity lifecycle from document issuance based on breeder documents, like birth certificates, to document usage and identity verification in various border crossing scenarios.
The D4FLY set of tools and systems investigated various technologies to enhance the quality of document and identity verification and lay the foundation for smooth on-the-move border crossing experiences for travellers. Novel sensor hardware based on advanced light field cameras and novel algorithms were developed in the project to enhance the verification accuracy via the combined usage of multiple different biometric modalities modalities, while performing under real-time constraints.
D4FLY created a document verification system that can verify a multitude of physical and electronic security features in travel documents like passports and help to analyse and detect fraud in breeder documents. The D4FLY developments included a border control kiosk geared with enhanced enrolment, verification and detection capabilities and a continuous on-the-move corridor for biometric verification. All innovations were validated against European societal values, fundamental rights, privacy, data protection and applicable legislation. Four different border control points and one document fraud expertise centre formed the project’s testing and demonstration grounds.
The main objectives of D4FLY were to identify existing or possible future fraud methods regarding travel documents and identity and to create new technologies in automated document fraud, spoofing and impostor attack detection. Further objectives include investigation of new technologies in person identification that improve the recognition of travellers on-the-move. The research was continuously monitored to be in compliance with European societal values, fundamental rights and applicable legislation.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Early in the project, high quality user needs were elicited, and the use cases have been refined in workshops with end users and stakeholders. Technical requirements have been derived and structured, based on the inputs from stakeholders and project participants. An overall system architecture was defined and implemented, specifying a modular, high performance system, using well defined communication interfaces, leveraging a distributed and heterogeneous development environment.
A novel 3D face recognition approach has been developed using images captured by innovative Raytrix cameras. Reconstruction of 3D face images from two 2D face images was researched and a new and improved approach was developed. A novel approach to thermal-to-visible face recognition was researched, where a thermal image could be captured during an enrolment process and verified against an image obtained in a capture area of a verification corridor. The research on iris-on-the-move technology using innovative Raytrix cameras led to an optimized camera setup for improved image quality and was tested in a prototype biometric corridor. A new algorithm for somatotype verification was developed and validated. All biometric components were required to perform under real-time constraints and were successfully integrated and tested in a biometric verification corridor prototype.
Alternative technologies for people identification and the application in the D4FLY scenarios have been explored. Research into techniques on continuous person verification using machine learning techniques based on smartphone sensors has led to the development of the first prototype smartphone application for enrolment and verification. Another prototype smartphone application was developed for traveller verification in confined spaces such as inside a coach, based on enrolled reference data using face recognition technologies. Potential use cases for blockchain technologies were investigated, and a prototype component was developed. A novel approach for determination of travellers’ risks based on fusion of meta data information collected from a variety of data sources was researched, and a prototype was created and discussed with end users.
A wide variety potential attacks to automated identity verification systems were analysed. Algorithms for the detection of spoofing attacks, like detection of manipulated images in travel documents, morphed images, presentation attacks and impostor fraud have been developed, integrated and tested.
New technologies in automated document fraud detection have been developed for breeder documents, as well as for travel documents (passports). Prototype software for the verification of document security elements were developed and integrated into the D4FLY document verification platform. A prototype for the analysis of breeder documents at detail level (e.g. investigation of stamps) and at tactical level has been developed and tested. Tools for travel pattern analysis based on stamp recognition in passports have been developed. A prototype for the application of blockchain technology in the context of document verification was developed.
All prototypes have been tested in field tests in the UK, Lithuania, Greece and the Netherlands. The valuable feedback from practitioners and end users was used to further improve the prototypes towards their final versions, which have been finally demonstrated in the UK (Highly automated border post and coach scenario) and in the Netherlands (Document verification).
A key part of the project work was to investigate the privacy, data protection, ethical and social issues raised in the context of the multimodal biometric systems. The ethics and data protection aspects have been continuously monitored throughout the research and development and the Ethical Impact Assessment was updated in the course of the project.
D4FLY disseminated its research and innovation results on its project web page and on multiple social media channels, through newsletters and blog posts, as well as through participation in conferences, workshops and other events.
h newsletters and blog posts, as well as through participation in conferences, workshops and other events.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

D4FLY progresses beyond state of the art in several ways and the results include:
1. Innovations in biometric recognition methods namely 3D-Face, Iris on-the-move, thermal-to-visible and somatotype verification
2. Advancements in counter spoofing and presentation attack detection technologies, especially liveliness detection technology and detection of manipulated images
3. Advancements in document verification technologies, extending the know-how on automatic detection and verification of document security elements.
4. New ways of using smartphones with apps for person identification as well as secure storage of traveller information.
Those technologies were tested in prototypes in four different scenarios at borders in four countries.
All activities were monitored and assessed for their compliance with the European data protection and privacy regulation, as well as ethical and societal impact.

d4fly-identity-lifecycle.png