SiMAX - The Sign Language Avatar

Project Information

SiMAX

Grant agreement ID: 778421

DOI

10.3030/778421

Project closed

EC signature date 17 August 2017

Start date 1 September 2017

End date 29 February 2020

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 1 563 372,50

EU contribution

€ 1 094 360,75

Coordinated by

Sign time GmbH
Austria

Periodic Reporting for period 2 - SiMAX (SiMAX - The Sign Language Avatar)

Reporting period: 2018-09-01 to 2020-02-29

Summary of the context and overall objectives of the project

Deaf people need sign language because they have difficulty understanding texts. 80% of deaf people are functionally illiterate.
In order to make information accessible to the deaf, companies, the media, public authorities, etc. must have it translated into sign language. Conventional methods of translation with interpreters in film studios are time-consuming and expensive.
Sign Time's goal is to develop a semi-automated sign language translation system and make it available as a cloud service. The translations are represented by an animated figure, an avatar in sign language.

Accessibility is an increasingly important value in society. With traditional methods, only a very small part of our digital information can be made accessible to the deaf.
With our technology it will be possible to translate mass content into sign language, as the costs are very low. In the long run it should be possible to translate almost any text into sign language. Since our system is adaptive, it becomes better with each application and thus faster and more economical. This will make it possible in the long term to make all digital content on the web accessible to the deaf. Worldwide.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Based on feedback from deaf people, a new avatar with special features was developed. This meets the requirements of the target group much better and receives very high approval values.

There were additionally 3 deaf persons trained to 3D animators. These produce the vocabulary of sign language. Since they use sign language themselves, they have better insight into this activity.
We have developed a quality management system for the creation of sign language vocabulary, which is managed by a deaf sign language translator. This ensures that our product meets the highest quality requirements at the module level.

Our existing translation software SiMAX was programmed into a first cloud-capable version.
Over 3,000 signs were animated as sign language vocabulary and are available for translation.

We implemented two pilot projects:
https://vimeo.com/263871219/e8c5efefde
https://vimeo.com/265404730

Dissemination:

We have presented our development at various international conferences and trade fairs, including the Mobile Word Congress in Barcelona, the Web Summit in Lisbon, the Innovator Summit in Berlin, trade fairs in Karlsruhe and Berlin, the RISE Conference in Hong Kong and many more. TV reports about us were shown in Austria and Hong Kong.
We have created and published GIFs animated by individual gestures. These can be used with different messengers like WhatsApp and Facebook Messenger like emoties. We got more than 270,000 hits so far. See https://giphy.com/simax

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

We have the world's first functioning system for the semi-automatic translation of text into animated sign language. The system will be further developed and adapted for different sign languages.
On the one hand, the aim is to make all available digital texts in sign languages accessible to deaf people all over the world. This will give them equal access to all information.
On the other hand, all companies, organisations, media and individuals should be able to communicate barrier-free in the interest of the deaf at very low cost. This is not only prescribed by law, it is also required by a modern society without discrimination.

Sign language happy

Sign language how are you

Sign language i love you