SignON - Sign Language Translation Mobile Application and Open Communications Framework

Project Information

SignON

Grant agreement ID: 101017255

DOI

10.3030/101017255

Project closed

EC signature date 30 November 2020

Start date 1 January 2021

End date 31 December 2023

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 5 616 718,75

EU contribution

€ 5 616 718,75

Coordinated by

DUBLIN CITY UNIVERSITY
Ireland

CORDIS provides links to public deliverables and publications of HORIZON projects.

Links to deliverables and publications from FP7 projects, as well as links to some specific result types such as dataset and software, are dynamically retrieved from OpenAIRE .

Deliverables

Other (12)

First Sign language-specific lexicon and structure
Second Sign language-specific lexicon and structure
Sign language recognition component and models

ign language recognition component and models

Final Sign language-specific lexicon and structure
Automatic speech recognition component and models
SignON platform and cloud infrastructure

Definition of the platform and cloud infrastructure for the project

First symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

Second symbolic intermediate representation - InterL-S

First symbolic intermediate representation - InterL-S

Linguistic description for ISL, BSL, VGT, NGT and LSE
First distributional intermediate representation based on embeddings - InterL-E
Second distributional intermediate representation based on embeddings - InterL-E
A hybrid intermediate representation

Documents, reports (26)

First Annual Ethical Report on Practices Towards the Stakeholder Communities

D921 Annual Ethical Report on Practices Towards the Stakeholder Communities

SignON Market Analysis
Second Annual Ethical Report on Internal Affairs

D9.3.2. Annual Ethical Report on Internal Affairs

Final Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.3 Annual Ethical Report on Practices Towards the Stakeholder Communities

First Annual reports on communication and dissemination activities
Final Annual reports on communication and dissemination activities
SignON Communication and Dissemination Plan
Quality Assurance plan
Report on the use-cases and usage domains and stakeholders' acceptance.

Report on use cases and acceptance

Case studies and evidence analysis

Description of case studies

Interim progress report

First Annual progress report

Report on Ethical approval

Report on the ethical approvals prepared, signed and collected during the project

First Interim Community building Report.

Report on community building

Second Impact Review
Final Annual Ethical Report on Internal Affairs

D9.3.3. Annual Ethical Report on Internal Affairs

Quality Assessment Report

Report on Quality of the sign on service

Ethical Guidelines and Protocols

D81 Ethical Guidelines and Protocols including an overview of how all data that will be processed is relevant and limited to the purposes of the SignOn project to ensure that we operate in accordance with the data minimisation principle M6

First Annual Ethical Report on Internal Affairs

D931 Annual Ethical Report on Internal Affairs

First Impact Review
Second Annual Ethical Report on Practices Towards the Stakeholder Communities

D9.2.2. Annual Ethical Report on Practices Towards the Stakeholder Communities

Final Impact Review
Final Report
Data Transfer Agreement

Agreement between all partners on the transference of data

Second Annual reports on communication and dissemination activities
Workshops, showcases, demonstrations report
Risk Management Plan

Demonstrators, pilots, prototypes (10)

Final release of the Communication Mobile Application

Final release of the Communication Mobile Application (demonstrator/code)

Second Natural Language processing pipeline
First release of the Open SignON Framework

First release of the framework (report + demonstrator)

First Natural Language Processing Pipelines
Final release of the Open SignON Framework

Final release of the Open SignON Framework (report + demonstrator)

Final version of virtual character
Intermediate release of the Open SignON Framework

Intermediate release of the Open SignON Framework (Report + demonstrator)

First release of the SignON Communication Mobile

First release of the SignON Communication Mobile (demonstrator/code)

First version of virtual character
Interactive co-creation web-based platform for learning from user input

Open Research Data Pilot (2)

Data Management Report

Report on Data management for the duration of the action

Data Management Plan

including a description of the technical and organisational measures that partners must implement to safeguard the rights and freedoms of research participants and the security measures that will be implemented to prevent unauthorised access to personal data or the equipment used for processing data

Publications

Other (7)

Onderzoek naar constituentenvolgorde in Vlaamse Gebarentaal. Enkelvoudige mededelende zinnen bij jongeren van 12 tot 18 jaar

Author(s): Lien Soetemans and Margot Janssens
Published in: Onderzoeksrapport KU Leuven, 2023
Publisher: N/A

Realising Inclusive Design Through Co-creation

Author(s): Brady, Aoife, Clarke, Emma, Dunne, Jane, Kearns, Anne, Leeson, Lorraine, Mathews, Elizabeth, Moiselle, Rachel, O’Boyle, Shaun, Omardeen, Rehana, Picron, Frankie, Shterionov, Dimitar, Van Landuyt, Davy
Published in: EARMA Conference, 2023
Publisher: EARMA

Design principles of an Automatic Speech Recognition functionality in a user-centric signed and spoken language translation system

Author(s): Aditya Parikh, Louis ten Bosch, Henk van den Heuvel, Cristian Tejedor-García
Published in: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Publisher: Tilburg University

Using co-creation to develop sign language machine translation technologies (Abstract)

Author(s): Shaun O’Boyle, Elizabeth Mathews, Davy Van Landuyt, Frankie Picron, Rehana Omardeen, Lorraine Leeson, Rachel Moiselle, Aoife Brady, Jorn Rijckaert, Caro Brosens, Anthony Ventresque, Ellen Rushe, Irene Murtagh, Andy Way, Dimitar Shterionov
Published in: Poster presentation at SCI:COM 2022, Dublin, Ireland, 2022
Publisher: N/A

Voorgetrainde Grote Taalmodellen en Automatische Vertaling.

Author(s): Vincent Vandeghinste and Tim Van de Cruys
Published in: Dixit. Tijdschrift voor Taaltechnologie, 2022
Publisher: Dixit

Progress, Challenges and Innovations of the SignON Project

Author(s): Vincent Vandeghinste, Mirella De Sisto, Dimitar Shterionov, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Scipioni, Aditya Parikh, Louis ten Bosch, John O'Flaherty, Joni Dambre and Jorn Rijckaert
Published in: Computational Linguistics in the Netherlands Conference (CLIN32), 2022
Publisher: Tilburg University

WordNets? SignNets!

Author(s): Ineke Schuurman, Vincent Vandeghinste, Caro Brosens, Margot Janssens and Thierry Declerck
Published in: Presented at the 32nd Meeting of Computational Linguistics in The Netherlands (CLIN 32), 2022
Publisher: CLIN

Conference proceedings (25)

A Data Augmentation and Pre-processing Technique for Sign Language Fingerspelling Recognition

Author(s): Frank Fowley, Ellen Rushe and Anthony Ventresque
Published in: 24th Irish Machine Vision and Image Processing Conference, 2022
Publisher: IMVIPC
DOI: 10.56541/xbav3102

BSL-Hansard: A parallel, multimodal corpus of English and interpreted British Sign Language data from parliamentary proceedings

Author(s): Euan McGill, Horacio Saggion.
Published in: Proceedings of the Second International Workshop on Automatic Translation for Signed and Spoken Languages, 2023
Publisher: European Association for Machine Translation

Isolated Sign Recognition from RGB Video using Pose Flow and Self-Attention

Author(s): Mathieu De Coster, Mieke Van Herreweghe, Joni Dambre
Published in: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021
Publisher: IEEE
DOI: 10.1109/cvprw53098.2021.00383

SignON: Bridging the gap between Sign and Spoken Languages

Author(s): Horacio Saggion, Dimitar Shterionov, Gorka Labaka, Tim Van de Cruys, Vincent Vandeghinste and Josep Blat
Published in: Proceedings of the Annual Conference of the Spanish Association for Natural Language Processing: Projects and Demonstrations (SEPLN-PD 2021) co-located with the Conference of the Spanish Society for Natural Language Processing (SEPLN 2021), Issue Annual, 2022, Page(s) 21-25
Publisher: CEUR Workshop Proceedings

SignON Sign Language Translation : Progress and Challenges

Author(s): Vincent Vandeghinste, Dimitar Shterionov, Mirella De Sisto, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Paolo Scipioni, Aditya Parikh, Louis ten Bosch, John O’Flaherty, Joni Dambre, Jorn Rijckaert, Bram Vanroy, Victor Ubieto Nogales, Santiago Egea Gomez, Ineke Schuurman, Gorka Labaka, Adrian Nunez-Marcos, Irene Murtagh, Euan McGill and Horacio Saggion
Published in: Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 2023, ISBN 9789520329471
Publisher: European Association for Machine Translation (EAMT)

Challenges with Sign Language Datasets for Sign Language Recognition and Translation.

Author(s): Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov and Horacio Saggion
Published in: Proceedings of the 13th International Conference on Language Resources and Evaluation (LREC), 2022
Publisher: European Language Resources Association

From Scarcity to Understanding: Transfer Learning for the Extremely Low Resource Irish Sign Language

Author(s): Ruth Holmes, Ellen Rushe, Mathieu De Coster, Maxim Bonnaerens, Shin'ichi Satoh, Akihiro Sugimoto, and Anthony Ventresque
Published in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023
Publisher: IEEE
DOI: 10.1109/iccvw60793.2023.00215

GoSt-ParC-Sign: Gold Standard Parallel Corpus of Sign and spoken language.

Author(s): Mirella De Sisto, Vincent Vandeghinste, Lien Soetemans, Caro Brosens, Dimitar Shterionov
Published in: Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 2023
Publisher: European Association for Machine Translation

Comparing Modular and End-To-End Approaches in ASR for Well-Resourced and Low-Resourced Languages

Author(s): Parikh, A, Ten Bosch, L., Van den Heuvel, H. & C. Tejedor Garcia
Published in: In: Proceedings of The 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023), 2023
Publisher: Association for Computational Linguistics

Part-of-Speech tagging Spanish Sign Language data and its applications in Sign Language machine translation

Author(s): Euan McGill, Luis Chiruzzo, Santiago Egea-Gómez, Horacio Saggion
Published in: Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains, 2023
Publisher: ACL

Frozen Pretrained Transformers for Neural Sign Language Translation.

Author(s): Mathieu De Coster, Karel D’Oosterlinck, Marija Pizurica, Paloma Rabaey, Severine Verlinden, Mieke Van Herreweghe, Joni Dambre.
Published in: Proceedings of the 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL), 2021
Publisher: MT Summit

A Linked Data Approach for linking and aligning Sign Language and Spoken Language Data.

Author(s): Thierry Declerck, Sam Bigeard, Fahad Khan, Irene Murtagh, Sussi Olsen, Mike Rosner, Ineke Schuurman, Andon Tchechmedjiev, and Andy Way
Published in: Proceedings of the Second International Workshop on Automatic Translation for Signed and Spoken Languages, 2023
Publisher: European Association for Machine Translation

Sign Language Machine Translation and the Sign Language Lexicon: A Linguistically Informed Approach

Author(s): Irene Murtagh, Víctor Ubieto Nogales, Josep Blat
Published in: In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), 2022
Publisher: Association for Machine Translation in the Americas

Sign Language Fingerspelling Recognition using Synthetic Data

Author(s): Frank Fowley and Anthony Ventresque
Published in: Proceedings of the 29th Irish Conference on Artificial Intelligence and Cognitive Science (AICS 2021), 2021
Publisher: AICS

Language Planning in Action: Depiction as a Driver of New Terminology in Irish Sign Language.

Author(s): Rachel Moiselle and Lorraine Leeson
Published in: Proceedings of the 10th Workshop on the Representation and Processing of Sign Languages (sign-lang@LREC 2022), 2022, Page(s) 139-143
Publisher: European Language Resources Association

Sign Language Translation: Ongoing Development, Challenges and Innovations in the SignON Project

Author(s): Dimitar Shterionov, Mirella De Sisto, Vincent Vandeghinste, Aoife Brady, Mathieu De Coster, Lorraine Leeson, Josep Blat, Frankie Picron, Marcello Paolo Scipioni, Aditya Parikh, Louis ten Bosch, John O’Flaherty, Joni Dambre, Jorn Rijckaert
Published in: Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, 2022
Publisher: European Association for Machine Translation

Trends and challenges for sign language recognition with machine learning

Author(s): Jérôme Fink, Mathieu De Coster, Joni Dambre, Benoît Frénay
Published in: Proceedings of the 2023 European Symposium on Artificial Neural Networks, 2023, ISBN 9782875870889
Publisher: ESANN
DOI: 10.14428/esann/2023.es2023-7

Improving Signer Independent Sign Language Recognition for Low Resource Languages. Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), pp. 45-52. Marseille, France.

Author(s): Ruth Holmes, Ellen Rushe, Frank Fowley, Anthony Ventresque
Published in: Proceedings of the 7th International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual: Challenges and Perspectives (SLTAT 2022), 2022, Page(s) 45-52
Publisher: European Language Resources Association

Syntax-aware Transformers for Neural Machine Translation: The Case of Text to Sign Gloss Translation.

Author(s): Santiago Egea Gómez, Euan McGill, Horacio Saggion
Published in: Proceedings of the 14th Workshop on Building and Using Comparable Corpora (BUCC2021), 2021, Page(s) 18-27
Publisher: INCOMA Ltd.

NGT-HoReCo and GoSt-ParC-Sign: Two new Sign Language - Spoken Language parallel corpora

Author(s): Mirella De Sisto, Dimitar Shterionov, Lien Soetemans, Vincent Vandeghinste, Caro Brosens
Published in: Proceedings of the CLARIN Annual Conference; 2023, 2023
Publisher: Krister Lindén, Jyrki Niemi, and Thalassia Kontino

Defining Meaningful Units: Challenges in Sign Segmentation and Segment-Meaning Mapping

Author(s): Mirella De Sisto, Dimitar Shterionov, Irene Murtagh, Myriam Vermeerbergen and Lorraine Leeson
Published in: 1st International Workshop on Automatic Translation for Signed and Spoken Languages (AT4SSL 2021), 2021, Page(s) 98-103
Publisher: online

Translating Spanish into Spanish Sign Language: Combining Rules and Data-driven Approaches

Author(s): Luis Chiruzzo, Euan McGill, Santiago Egea-Gómez, Horacio Saggion
Published in: Proceedings of the 29th International Conference on Computational Linguistics, 2022
Publisher: International Committee on Computational Linguistics

Querying A Sign Language Dictionary with Videos Using Dense Vector Search

Author(s): Mathieu De Coster and Joni Dambre
Published in: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023
Publisher: IEEE

When shared space and time don’t matter. Remote video-mediated (synchronous and asynchronous ) communication in Flemish Sign Language.

Author(s): Lien Soetemans and Myriam Vermeerbergen
Published in: 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2023
Publisher: IEEE
DOI: 10.1109/icasspw59220.2023.10193316

Attention Analysis of a Sign Language Recognition Task on the AUTSL Dataset

Author(s): Jeanne Coppin, Mathieu De Coster, Joni Dambre
Published in: 32nd Meeting of Computational Linguistics in The Netherlands (CLIN32), Issue Annual, 2022
Publisher: Tilburg University

Peer reviewed articles (1)

Leveraging Frozen Pretrained Written Language Models for Neural Sign Language Translation

Author(s): Mathieu De Coster, Joni Dambre
Published in: Information, 2022, ISSN 2078-2489
Publisher: Multidisciplinary Digital Publishing Institute (MDPI)
DOI: 10.3390/info13050220

Book chapters (1)

Linguistically Enhanced Text to Sign Gloss Machine Translation

Author(s): Egea Gómez, S., Chiruzzo, L., McGill, E., Saggion, H.
Published in: Lecture Notes in Computer Science, vol 13286, 2022
Publisher: Springer
DOI: 10.1007/978-3-031-08473-7_16