Accessible Voice-Activated Technologies in a Voice-First World

Project Information

NUVOIC

Grant agreement ID: 950872

DOI

10.3030/950872

Project closed

EC signature date 29 April 2020

Start date 1 May 2020

End date 30 April 2023

Funded under

• Horizon 2020 Framework Programme

Total cost

€ 3 620 421,00

EU contribution

€ 2 620 691,00

Coordinated by

TECHNOLOGIES OF VOICE INTERFACE LTD
Israel

Periodic Reporting for period 3 - NUVOIC (Accessible Voice-Activated Technologies in a Voice-First World)

Reporting period: 2022-05-01 to 2023-04-30

Summary of the context and overall objectives of the project

Speech impairments may range from mild to severe and refer to an impaired ability to produce speech sounds. People with speech and motor disabilities, as well as elderly with non-standard speech patterns, struggle daily to communicate with people around them. The NUVOIC project benefits people with speech and motor disabilities, and elderly with non-standard speech patterns, by providing the affected individuals an effective means to communicate with the people around them, and increased independence by enabling access to mainstream voice-driven devices and technologies.
The overall objective of the project was therefore to deliver to the market a novel core speech recognition technology designed to recognize impaired speech. The Nuvoic project follows on the success of Voiceit’s previous SME Instrument project, in which Voiceitt launched its first-generation discrete speech recognition technology, which helps people with severe speech and motor impairments to communicate by voice in any language with a pre-calibrated set of phrases; a unique integration with Amazon Alexa enabled people with speech and motor impairs to perform basic tasks independently by voice, for the first time. Thus, Nuvoic was launched with the objective of developing Voiceitt’s next generation technology, which would expand Voiceitt’s capabilities with more flexible algorithms which enables spontaneous speech after the user calibrates the software. This algorithmic achievement would be achieved through extensive audio data collection and validation trials with individuals with a variety of underlying medical and age-related conditions affecting speech. Moreover, new commercial use cases for the core speech technology would be explored, validated and pursued through strategic partnerships with global corporate technology organisations, strengthening Voiceitt’s positioning as leading provider of speech recognition for non-standard speech and leveraging the cutting edge voice AI for greater impact through more accessible and inclusive societies. With product launches slated for 2022 and 2023, the project provided affected individuals an innovative technological means to communicate with the people around them, as well as access to the benefits of mainstream voice-driven devices and technologies. The project created commercial added value by inherently advancing Voiceitt’s novel core speech recognition technology for impaired speech. The strengthened speech technology in the new products enbles people with motor, cognitive and speech-related challenges to enjoy the benefits of mainstream voice-driven technologies, thus improving independence and quality of life for these individuals. The overall progress in the project has exceeded the partners original expectations both in the context of the technology development and the commercial opportunities. The ongoing customer engagement and feedback during each stage of Nuvoic’s development ensures that the final outcomes address the needs of the end users, and support the business case and successful adoption of the Nuvoic commercial product.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Participants with non-standard speech correlated with a variety of underlying conditions and disorders have been recruited to participate in the Nuvoic data collection and testing activities. The proper procedures and principles of data privacy, ethical research, and informed consent have been followed. Recorded speech data has been collected from Nuvoic participants to support the accelerated development of the core Nuvoic speech technology software, resulting in a large, unique database of non-standard speech. Customer research was completed to evaluate how the Nuvoic voice technology integrated with Nuvoic Home and Nuvoic Shop products can improve elderly and disabled consumers’ lives. Prototype of the Nuvoic Home and Nuvoic Shop were designed, which integrates Voiceitt’s technology within new use cases.
The main results of the project include technological, commercial, and impact milestones. Voiceitt’s next-generation continuous speech recognition technology was launched in 2023 following extensive engagement with end users and stakeholders in UK and Europe through data collection that augmented Voiceitt’s AI datasets. The technology was launched as a browser-based application and an API, which enables the technology to be integrated with mainstream products making voice-enabled devices, platforms, and software accessible for people with disabilities. In exploring significant use cases, prototypes for smart home and speaker speaker applications and an e-commerce app were successfully developed and their value investigated and evaluated through market research, interviews, and testing with end users.
Consortium partners leveraged their networks and industry knowledge to reach out and encourage widespread dissemination of communication material and project results/outcomes. To this end, the consortium performed profound engagement efforts, both for the user community, the professional community, and, importantly, a B2B customer base (e.g. organisations serving people with disabilities and elderly). The continuation of the engagement efforts has been driven by a communication and social media strategy to:
*Establish stronger users’ sharing on social media and stories;
*Educate about speech disability to the network that supports the users: families, friends, caregivers, and teachers;
*Communicate through relevant forums and social media channels;
*Engage the professional community to support our technology and efforts, including presentations at relevant conferences and forums, connections with educational institutions and non-profit organisations who serve people with disabilities.
*Information packages/materials developed and distributed to consumers, influencers and advocates for Nuvoic products;
*Publications of press releases/trial results in scientific journals or industry blogs/news media.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

This project concludes at an auspicious time, in which the industry of Voice AI grows in significance as one of the most dynamic niches which conversational / generative AI and the interest among corporations in inclusivity and diversity significantly increases. Both of these significant industry trends expand both the commercial potential for the next-generation Voiceitt technology, when integrated as an API or sold in bulk licenses to companies committed to being competitive within the voice space and also ensuring new digital products are accessible to all customers. Voiceitt’s next generation technology exceeds state of the art capabilities in supporting highly dysarthric speech patterns after relatively short training and calibration of the software through submission of voice recordings over 1-3 hours (the timing for which continues to decrease, and at the time of this report it is estimated that eight months out, little or not training may be needed at all!). Voiceitt’s first high-profile integration of the next generation technology was announced in May 2023: its integration with the global collaboration platform, Webex, enabling people with speech and motor disabilities to communicate and be understood in video meetings. The continued interest from disability organisations, governments, research institutions, healthcare entities, and technology partners sets the foundation for further success and impact in the EU and worldwide.

jenny-1.jpg