Solutions for voice interaction towards natural crew assistant

Informazioni relative al progetto

VOICI

ID dell’accordo di sovvenzione: 785401

DOI

10.3030/785401

Progetto chiuso

Data della firma CE 20 Febbraio 2018

Data di avvio 1 Marzo 2018

Data di completamento 29 Febbraio 2020

Finanziato da

• SOCIETAL CHALLENGES - Smart, Green And Integrated Transport

Costo totale

€ 1 250 750,00

Contributo UE

€ 1 111 250,00

Coordinato da

SINTEF AS
Norway

Periodic Reporting for period 2 - VOICI (Solutions for voice interaction towards natural crew assistant)

Periodo di rendicontazione: 2019-03-01 al 2020-02-29

Sintesi del contesto e degli obiettivi generali del progetto

"VOICI has aimed to develop a natural crew assistant – designed to process natural spoken requests from the crew members and help them perform complex actions. Such an assistant can reduce the cognitive workload associated with the increasingly complex human-machine interfaces in modern aircraft cockpits.
VOICI's main goal has been to demonstrate the natural crew assistant at Technology Readiness Level 3, i.e. experimental proof of concept. This includes speech capture, speech recognition and interpretation, and speech synthesis for dialog with the crew. An Audio Evaluation Environment has been established, for developing and testing the crew assistant under realistic audio/noise conditions.
The project is part of the ""extended cockpit"" work package under Clean Sky 2's ""Systems"" technology demonstration. By reducing crew workload VOICI contributes to optimization of operations; flight safety, crew awareness, better maintenance, reduced cost of operations and generally higher efficiency and lower stress.
The consortium comprises 4 partners: SINTEF (Norway, R&D institute, coordinator), sensiBel (Norway, SME), Multitel (Belgium, R&D institute), Acapela (Belgium, SME).
Thales (France, Clean Sky 2 partner) is Topic Manager.
The project has run 24 months, ending February 2020. By the project end the main goal has been achieved - providing proof of concept demonstration of the crew assistant for two use cases selected by the Topic Manager. The project has contributed to innovation and job creation by improving component technologies, demonstrating the concept and establishing new cooperation within the consortium and with the Topic Manager."

Lavoro eseguito dall’inizio del progetto fino alla fine del periodo coperto dalla relazione e principali risultati finora ottenuti

VOICI has established an Audio Evaluation Environment, emulating a case selected by the Topic Manager: a Dassault Falcon 2000 cockpit. This includes a geometrical cockpit model and calibrated noise recording, split into multiple channels so as to represent multiple noise sources. Hence, a realistic noise and reverberation environment has been created, including its directional properties. A digital version of the environment has been established by measuring impulse responses between multiple loudspeakers and multiple microphones.

Two methods of speech capture have been investigated: A pilot's headset using a close-talk microphone and an ambient microphone array. Captured speech has been processed by stand-alone speech/dialog processing functions, i.e. suitable for embedding in a cockpit without connection to external, e.g. cloud-based services. The processing includes speech enhancement, voice activity detection, context dependent speech recognition, dialog system and text-to-speech for the noisy environment.

Due to the high noise levels, the close-talk microphone has been found to be the preferred technology for speech capture. Still, arrays of SME partner sensiBel's very low noise optical microphones were shown to produce significant noise suppression when used with modern beamforming algorithms. VOICI has contributed to sensiBel's microphone technology development, representing a significant project exploitation potential both in aviation/transport and in a much wider market.

The speech processing engine has demonstrated 5% Word Error Rate for cruising flight conditions, relying on an context -dependent language model and deep learning approaches for training. The dialog system was built to additionally include touch-screen input, together with external aviation information such as airport and weather data. It has been designed for easy inclusion of extended skills in terms of new stories, intent, entities and actions. Speech based feedback to the pilot was implemented by SME partner Acapela's text-to-speech function optimized for the noisy environment.
The total crew assistant has been called SOLO. The Topic Manager has signalled that an Implementation agreement will be negotiated. Acapela has elaborated plans for exploitation of both the total concept and the optimized text-to-speech function.

Dissemination items of the project have been identified. Due to the relatively short duration of the project, most of the dissemination-plan will be implemented after the end of the project. This will take place in close dialog with the Topic Manager.

Progressi oltre lo stato dell’arte e potenziale impatto previsto (incluso l’impatto socioeconomico e le implicazioni sociali più ampie del progetto fino ad ora)

"VOICI's total concept has contributed to the state-of-the-art in cockpit-embedded speech processing and dialog systems – tailored to the aviation scenario and high noise levels. Similarly, the project has contributed to the state-of-the-art in component technologies of the SME partners; very low noise optical microphones and noise-optimized text-to-speech technology.

Through a reduced crew workload VOICI will contribute to Clean Sky 2's ""extended cockpit"" goals; optimization of operations, flight safety, crew awareness, better maintenance, reduced cost of operations and generally higher efficiency and lower stress."

SOLO architecture

Embedded, standalone natural crew assistant

Microphone array with front end electronics. Endfire configuration example.

Audio evaluation environment built from sheet plywood and transparent acrylic