from Intrinsic Motivations to Transitional Wearable INtelligent companions for autism spectrum disorder

Project Information

IM-TWIN

Grant agreement ID: 952095

DOI

10.3030/952095

Project closed

EC signature date 28 May 2020

Start date 1 November 2020

End date 31 October 2023

Funded under

• EXCELLENT SCIENCE - Future and Emerging Technologies (FET)

Total cost

€ 1 999 965,00

EU contribution

€ 1 999 965,00

Coordinated by

CONSIGLIO NAZIONALE DELLE RICERCHE
Italy

Periodic Reporting for period 3 - IM-TWIN (from Intrinsic Motivations to Transitional Wearable INtelligent companions for autism spectrum disorder)

Reporting period: 2022-11-01 to 2023-10-31

Summary of the context and overall objectives of the project

The IM-TWIN project aims to develop some of the outcomes of a previous EU basic research project, “GOAL-Robots”, towards market exploitation. GOAL-Robots aimed to study how intrinsic motivations (“curiosity”) drive exploration and learning in children, and how such processes can be used to develop innovative autonomous robots. This led to the conception of the idea that intrinsic motivations can be used to build engaging interactive robots usable for the treatment of children with developmental disorders, in particular within the Autism Spectrum Disorder (ASD). ASD is a neurodevelopmental condition with dramatic importance for the well-being of society: recent reviews about epidemiology show how a significant part of the population in developed countries is diagnosed with this condition (estimates range from 1 in 67 to 1 in 132). Led by this idea, we developed a “wearable companion robot” usable for the early treatment of ASD, called “PlusMe”, now at the prototype stage. This prototype was further developed within a second EU project called “PlusMe: Transitional wearable companions for the therapy with children with Autism Spectrum Disorders” (as the prototype), starting earlier and now running in parallel with IM-TWIN (PlusMe).

In continuation with these initiatives, the IM-TWIN project is carrying out two sets of objectives. The first is to develop a highly-modular system pivoting on the PlusMe, called “IM-TWIN” (as the project itself), addressing the needs of the market segment involving ASD children, therapy centres and, potentially, families of ASD children. The new IM-TWIN system involves different components and implements different functionalities: (a) integrating the PlusMe with biosensors, in particular by embedding them into a new sensorised T-shirt worn by the children for detecting the child’s affective/emotional states; (b) embedding cameras in the environment, for face expression recognition; (c) endowing the PlusMe with basic intelligent behaviour; (d) and integrating all components as a whole IoT system that allows the parallel synchronised collection of data from the users, usable for on-line and off-line screening, monitoring, therapy, and research. The second set of objectives aims to: (a) validate the device and its components with target stakeholders: ASD children under therapy, therapy centres, and families.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Together these activities are aiming to advance the Technological Readiness Level of the IM-TWIN components, in particular the PlusMe prototype to TRL7. This advancement is based on both the technological developments and validation activities illustrated above, and on a number of exploitation and dissemination activities. The latter activities involve the identification of the target groups, the analyses of ASD-related markets, the refinement and implementation of an effective IPR strategy, the steps for individual-partner and consortium exploitation of results, and the evaluation of the possible final creation of a startup for the exploitation of the IM-TWIN system and components.

The project has thus far achieved notable steps towards the aforementioned objectives, in particular it has: (a) redesigned and industrialised the PlusMe, endowed it with new software control, new functionalities, a higher robustness, and a new software for easier use; (b) started the test the PlusMe with ASD children; (c) designed and started to implement the new sensorised T-shirt; (d) designed and started to test the sensors and software of the sensorised T-shirt; (e) implemented the hardware system to support face expression recognition; (f) implemented the HW/SW communication system linking all components of the IM-TWIN system; (g) developed a new state-of-the-art technique to process signals (fast Continuous Wavelet Transform); (h) managed the ethical clearance and data protection practises; (i) designed the experimental protocol to test the T-shirt.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The PlusMe represents a novel "Transitional Wearable Companion", a sensorised "robot" that can be used to support innovative therapies of children with ASD. Its integration with the sensorised T-shirt and environment cameras forming the whole IoT IM-TWIN system will allow having an innovative modular flexible tool for the massive collection of data usable for monitornig ASD children, supporting their therapy, and fostering further scientific research. At the same time, various sub-set of the IM-TWIN system components have potential for the market, such as the PlusMe usable as an intelligent toy for families, the sensorised T-shirt for physiological state recognition, and the M-TWIN system for data-collection relevant for therapeutic centers. The further development of these devices in the project thus foresees a relevant delivery of economic and social value for the market and wider society.

IM-TWIN system

PlusMe soft interactive toy