SignON - Sign Language Translation Mobile Application and Open Communications Framework

Projektinformationen

SignON

ID Finanzhilfevereinbarung: 101017255

DOI

10.3030/101017255

Projekt abgeschlossen

EK-Unterschriftsdatum 30 November 2020

Startdatum 1 Januar 2021

Enddatum 31 Dezember 2023

Finanziert unter

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Gesamtkosten

€ 5 616 718,75

EU-Beitrag

€ 5 616 718,75

Koordiniert durch

DUBLIN CITY UNIVERSITY
Ireland

CORDIS bietet Links zu öffentlichen Ergebnissen und Veröffentlichungen von HORIZONT-Projekten.

Links zu Ergebnissen und Veröffentlichungen von RP7-Projekten sowie Links zu einigen Typen spezifischer Ergebnisse wie Datensätzen und Software werden dynamisch von OpenAIRE abgerufen.

Leistungen

Other (12)

First Sign language-specific lexicon and structure
Second Sign language-specific lexicon and structure
Sign language recognition component and models

ign language recognition component and models

Final Sign language-specific lexicon and structure
Automatic speech recognition component and models
SignON platform and cloud infrastructure

Definition of the platform and cloud infrastructure for the project

First symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

First symbolic intermediate representation - InterL-S

Linguistic description for ISL, BSL, VGT, NGT and LSE
First distributional intermediate representation based on embeddings - InterL-E
Second distributional intermediate representation based on embeddings - InterL-E
A hybrid intermediate representation

Documents, reports (26)

First Annual Ethical Report on Practices Towards the Stakeholder Communities

D921 Annual Ethical Report on Practices Towards the Stakeholder Communities

SignON Market Analysis
Second Annual Ethical Report on Internal Affairs

D9.3.2. Annual Ethical Report on Internal Affairs

Final Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.3 Annual Ethical Report on Practices Towards the Stakeholder Communities

First Annual reports on communication and dissemination activities
Final Annual reports on communication and dissemination activities
SignON Communication and Dissemination Plan
Quality Assurance plan
Report on the use-cases and usage domains and stakeholders' acceptance.

Report on use cases and acceptance

Case studies and evidence analysis

Description of case studies

Interim progress report

First Annual progress report

Report on Ethical approval

Report on the ethical approvals prepared, signed and collected during the project

First Interim Community building Report.

Report on community building

Second Impact Review
Final Annual Ethical Report on Internal Affairs

D9.3.3. Annual Ethical Report on Internal Affairs

Quality Assessment Report

Report on Quality of the sign on service

Ethical Guidelines and Protocols

D81 Ethical Guidelines and Protocols including an overview of how all data that will be processed is relevant and limited to the purposes of the SignOn project to ensure that we operate in accordance with the data minimisation principle M6

First Annual Ethical Report on Internal Affairs

D931 Annual Ethical Report on Internal Affairs

First Impact Review
Second Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.2. Annual Ethical Report on Practices Towards the Stakeholder Communities

Final Impact Review
Final Report
Data Transfer Agreement

Agreement between all partners on the transference of data

Second Annual reports on communication and dissemination activities
Workshops, showcases, demonstrations report
Risk Management Plan

Demonstrators, pilots, prototypes (10)

Final release of the Communication Mobile Application

Final release of the Communication Mobile Application (demonstrator/code)

Second Natural Language processing pipeline
First release of the Open SignON Framework

First release of the framework (report + demonstrator)

First Natural Language Processing Pipelines
Final release of the Open SignON Framework

Final release of the Open SignON Framework (report + demonstrator)

Final version of virtual character
Intermediate release of the Open SignON Framework

Intermediate release of the Open SignON Framework (Report + demonstrator)

First release of the SignON Communication Mobile

First release of the SignON Communication Mobile (demonstrator/code)

First version of virtual character
Interactive co-creation web-based platform for learning from user input

Open Research Data Pilot (2)

Data Management Report

Report on Data management for the duration of the action

Data Management Plan

including a description of the technical and organisational measures that partners must implement to safeguard the rights and freedoms of research participants and the security measures that will be implemented to prevent unauthorised access to personal data or the equipment used for processing data

Veröffentlichungen

Other (7)

Onderzoek naar constituentenvolgorde in Vlaamse Gebarentaal. Enkelvoudige mededelende zinnen bij jongeren van 12 tot 18 jaar

Autoren: Lien Soetemans and Margot Janssens
Veröffentlicht in: Onderzoeksrapport KU Leuven, 2023
Herausgeber: N/A

Realising Inclusive Design Through Co-creation

Autoren: Brady, Aoife, Clarke, Emma, Dunne, Jane, Kearns, Anne, Leeson, Lorraine, Mathews, Elizabeth, Moiselle, Rachel, O’Boyle, Shaun, Omardeen, Rehana, Picron, Frankie, Shterionov, Dimitar, Van Landuyt, Davy
Veröffentlicht in: EARMA Conference, 2023
Herausgeber: EARMA

Design principles of an Automatic Speech Recognition functionality in a user-centric signed and spoken language translation system

Autoren: Aditya Parikh, Louis ten Bosch, Henk van den Heuvel, Cristian Tejedor-García
Veröffentlicht in: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Herausgeber: Tilburg University

Using co-creation to develop sign language machine translation technologies (Abstract)

Autoren: Shaun O’Boyle, Elizabeth Mathews, Davy Van Landuyt, Frankie Picron, Rehana Omardeen, Lorraine Leeson, Rachel Moiselle, Aoife Brady, Jorn Rijckaert, Caro Brosens, Anthony Ventresque, Ellen Rushe, Irene Murtagh, Andy Way, Dimitar Shterionov
Veröffentlicht in: Poster presentation at SCI:COM 2022, Dublin, Ireland, 2022
Herausgeber: N/A

Voorgetrainde Grote Taalmodellen en Automatische Vertaling.

Autoren: Vincent Vandeghinste and Tim Van de Cruys
Veröffentlicht in: Dixit. Tijdschrift voor Taaltechnologie, 2022
Herausgeber: Dixit

Progress, Challenges and Innovations of the SignON Project

Autoren: Vincent Vandeghinste, Mirella De Sisto, Dimitar Shterionov, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Scipioni, Aditya Parikh, Louis ten Bosch, John O'Flaherty, Joni Dambre and Jorn Rijckaert
Veröffentlicht in: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Herausgeber: Tilburg University

WordNets? SignNets!

Autoren: Ineke Schuurman, Vincent Vandeghinste, Caro Brosens, Margot Janssens and Thierry Declerck
Veröffentlicht in: Presented at the 32nd Meeting of Computational Linguistics in The Netherlands (CLIN 32), 2022
Herausgeber: CLIN

Conference proceedings (25)

A Data Augmentation and Pre-processing Technique for Sign Language Fingerspelling Recognition

Autoren: Frank Fowley, Ellen Rushe and Anthony Ventresque
Veröffentlicht in: 24th Irish Machine Vision and Image Processing Conference, 2022
Herausgeber: IMVIPC
DOI: 10.56541/xbav3102

BSL-Hansard: A parallel, multimodal corpus of English and interpreted British Sign Language data from parliamentary proceedings

Autoren: Euan McGill, Horacio Saggion.
Veröffentlicht in: Proceedings of the Second International Workshop on Automatic Translation for Signed and Spoken Languages, 2023
Herausgeber: European Association for Machine Translation

Isolated Sign Recognition from RGB Video using Pose Flow and Self-Attention

Autoren: Mathieu De Coster, Mieke Van Herreweghe, Joni Dambre
Veröffentlicht in: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Herausgeber: IEEE
DOI: 10.1109/cvprw53098.2021.00383

SignON: Bridging the gap between Sign and Spoken Languages

Autoren: Horacio Saggion, Dimitar Shterionov, Gorka Labaka, Tim Van de Cruys, Vincent Vandeghinste and Josep Blat
Veröffentlicht in: Proceedings of the Annual Conference of the Spanish Association for Natural Language Processing: Projects and Demonstrations (SEPLN-PD 2021) co-located with the Conference of the Spanish Society for Natural Language Processing (SEPLN 2021), Ausgabe Annual, 2022, Seite(n) 21-25
Herausgeber: CEUR Workshop Proceedings

SignON Sign Language Translation : Progress and Challenges

Autoren: Vincent Vandeghinste, Dimitar Shterionov, Mirella De Sisto, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Paolo Scipioni, Aditya Parikh, Louis ten Bosch, John O’Flaherty, Joni Dambre, Jorn Rijckaert, Bram Vanroy, Victor Ubieto Nogales, Santiago Egea Gomez, Ineke Schuurman, Gorka Labaka, Adrian Nunez-Marcos, Irene Murtagh, Euan McGill and Horacio Saggion
Veröffentlicht in: Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 2023, ISBN 9789520329471
Herausgeber: European Association for Machine Translation (EAMT)

Challenges with Sign Language Datasets for Sign Language Recognition and Translation.

Autoren: Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov and Horacio Saggion
Veröffentlicht in: Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC), 2022
Herausgeber: European Language Resources Association

From Scarcity to Understanding: Transfer Learning for the Extremely Low Resource Irish Sign Language

Autoren: Ruth Holmes, Ellen Rushe, Mathieu De Coster, Maxim Bonnaerens, Shin'ichi Satoh, Akihiro Sugimoto, and Anthony Ventresque
Veröffentlicht in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023
Herausgeber: IEEE
DOI: 10.1109/iccvw60793.2023.00215

GoSt-ParC-Sign: Gold Standard Parallel Corpus of Sign and spoken language.

Autoren: Mirella De Sisto, Vincent Vandeghinste, Lien Soetemans, Caro Brosens, Dimitar Shterionov
Veröffentlicht in: Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 2023
Herausgeber: European Association for Machine Translation

Comparing Modular and End-To-End Approaches in ASR for Well-Resourced and Low-Resourced Languages

Autoren: Parikh, A, Ten Bosch, L., Van den Heuvel, H. & C. Tejedor Garcia
Veröffentlicht in: In: Proceedings of The 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023), 2023
Herausgeber: Association for Computational Linguistics

Part-of-Speech tagging Spanish Sign Language data and its applications in Sign Language machine translation

Autoren: Euan McGill, Luis Chiruzzo, Santiago Egea-Gómez, Horacio Saggion
Veröffentlicht in: Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains, 2023
Herausgeber: ACL

Frozen Pretrained Transformers for Neural Sign Language Translation.

Autoren: Mathieu De Coster, Karel D’Oosterlinck, Marija Pizurica, Paloma Rabaey, Severine Verlinden, Mieke Van Herreweghe, Joni Dambre.
Veröffentlicht in: Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL), 2021
Herausgeber: MT Summit

A Linked Data Approach for linking and aligning Sign Language and Spoken Language Data.

Autoren: Thierry Declerck, Sam Bigeard, Fahad Khan, Irene Murtagh, Sussi Olsen, Mike Rosner, Ineke Schuurman, Andon Tchechmedjiev, and Andy Way
Veröffentlicht in: Proceedings of the Second International Workshop on Automatic Translation for Signed and Spoken Languages, 2023
Herausgeber: European Association for Machine Translation

Sign Language Machine Translation and the Sign Language Lexicon: A Linguistically Informed Approach

Autoren: Irene Murtagh, Víctor Ubieto Nogales, Josep Blat
Veröffentlicht in: In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), 2022
Herausgeber: Association for Machine Translation in the Americas

Sign Language Fingerspelling Recognition using Synthetic Data

Autoren: Frank Fowley and Anthony Ventresque
Veröffentlicht in: Proceedings of the 29th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2021), 2021
Herausgeber: AICS

Language Planning in Action: Depiction as a Driver of New Terminology in Irish Sign Language.

Autoren: Rachel Moiselle and Lorraine Leeson
Veröffentlicht in: Proceedings of the 10th Workshop on the Representation and Processing of Sign Languages (sign-lang@LREC 2022), 2022, Seite(n) 139-143
Herausgeber: European Language Resources Association

Sign Language Translation: Ongoing Development, Challenges and Innovations in the SignON Project

Autoren: Dimitar Shterionov, Mirella De Sisto, Vincent Vandeghinste, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Paolo Scipioni, Aditya Parikh, Louis ten Bosch, John O’Flaherty, Joni Dambre, Jorn Rijckaert
Veröffentlicht in: Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, 2022
Herausgeber: European Association for Machine Translation

Trends and challenges for sign language recognition with machine learning

Autoren: Jérôme Fink, Mathieu De Coster, Joni Dambre, Benoît Frénay
Veröffentlicht in: Proceedings of the 2023 European Symposium on Artificial Neural Networks, 2023, ISBN 9782875870889
Herausgeber: ESANN
DOI: 10.14428/esann/2023.es2023-7

Improving Signer Independent Sign Language Recognition for Low Resource Languages. Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), pp. 45-52. Marseille, France.

Autoren: Ruth Holmes, Ellen Rushe, Frank Fowley, Anthony Ventresque
Veröffentlicht in: Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), 2022, Seite(n) 45-52
Herausgeber: European Language Resources Association

Syntax-aware Transformers for Neural Machine Translation: The Case of Text to Sign Gloss Translation.

Autoren: Santiago Egea Gómez, Euan McGill, Horacio Saggion
Veröffentlicht in: Proceedings of the 14th Workshop on Building and Using Comparable Corpora (BUCC2021), 2021, Seite(n) 18-27
Herausgeber: INCOMA Ltd.

NGT-HoReCo and GoSt-ParC-Sign: Two new Sign Language - Spoken Language parallel corpora

Autoren: Mirella De Sisto, Dimitar Shterionov, Lien Soetemans, Vincent Vandeghinste, Caro Brosens
Veröffentlicht in: Proceedings of the CLARIN Annual Conference; 2023, 2023
Herausgeber: Krister Lindén, Jyrki Niemi, and Thalassia Kontino

Defining Meaningful Units: Challenges in Sign Segmentation and Segment-Meaning Mapping

Autoren: Mirella De Sisto, Dimitar Shterionov, Irene Murtagh, Myriam Vermeerbergen and Lorraine Leeson
Veröffentlicht in: 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL 2021), 2021, Seite(n) 98-103
Herausgeber: online

Translating Spanish into Spanish Sign Language: Combining Rules and Data-driven Approaches

Autoren: Luis Chiruzzo, Euan McGill, Santiago Egea-Gómez, Horacio Saggion
Veröffentlicht in: Proceedings of the 29th International Conference on Computational Linguistics, 2022
Herausgeber: International Committee on Computational Linguistics

Querying A Sign Language Dictionary with Videos Using Dense Vector Search

Autoren: Mathieu De Coster and Joni Dambre
Veröffentlicht in: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023
Herausgeber: IEEE

When shared space and time don’t matter. Remote video-mediated (synchronous and asynchronous ) communication in Flemish Sign Language.

Autoren: Lien Soetemans and Myriam Vermeerbergen
Veröffentlicht in: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023
Herausgeber: IEEE
DOI: 10.1109/icasspw59220.2023.10193316

Attention Analysis of a Sign Language Recognition Task on the AUTSL Dataset

Autoren: Jeanne Coppin, Mathieu De Coster, Joni Dambre
Veröffentlicht in: 32nd Meeting of Computational Linguistics in The Netherlands (CLIN32), Ausgabe Annual, 2022
Herausgeber: Tilburg University

Peer reviewed articles (1)

Leveraging Frozen Pretrained Written Language Models for Neural Sign Language Translation

Autoren: Mathieu De Coster, Joni Dambre
Veröffentlicht in: Information, 2022, ISSN 2078-2489
Herausgeber: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/info13050220

Book chapters (1)

Linguistically Enhanced Text to Sign Gloss Machine Translation

Autoren: Egea Gómez, S., Chiruzzo, L., McGill, E., Saggion, H.
Veröffentlicht in: Lecture Notes in Computer Science, vol 13286, 2022
Herausgeber: Springer
DOI: 10.1007/978-3-031-08473-7_16