Periodic Reporting for period 2 - COUCH (Council of Coaches) Reporting period: 2019-03-01 to 2020-08-31 Summary of the context and overall objectives of the project Despite the proliferation of ICT solutions for personalized healthcare, there is still no easy way to provide older adults with integrated coaching services. Council of Coaches (COUCH) introduces a radically new virtual coaching concept based on multiple autonomous, embodied virtual coaches. Together, they are a personal council that meets the needs of older adults in an integrated way. Each Coach has his own expertise, personality, and style of coaching; they might not always agree with each other, but they all share a single goal: to support you across every aspect of well-being, including physical, social, cognitive and mental support. The Council members listen to you, inform you, help you to set personal goals and inspire you to take control of your health. Give the council your thoughts, or listen and observe how the individual Coaches exchange their views on numerous issues. Apply your new skills in daily life and if the need arises, contact any of the coaches anytime, anywhere.COUCH resulted in the open-source Agents United Platform (www.agents-united.org) providing core functionalities for the development and deployment of virtual agents, including technologies for non-obtrusive interactions between the user agents. The platform enables developers and innovators to introduce new multi-agent based (coaching) services. In addition, COUCH has demonstrated how the concept of a virtual council of coaches keeps users engaged in working on their healthy lifestyle through a robust, content-driven functional demonstrator. Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far A total of four iterations of technical- and functional demonstrators have been developed, based on user input, and evaluated with end-users. An RRI vision was established in how the Council of Coaches project handles ethical issues within a project framework and how to develop a coaching application within ethical, legal and societal boundaries.The main outcomes of the project are its two core demonstrators – one “Functional Demonstrator”, that served to demonstrate the proof of concept of the Council of Coaches concept, and a second “Technical Demonstrator” that served to push the boundaries of state-of-the-art embodied conversational agent systems.The Functional Demonstrator is a stable, highly scalable web application. This application underwent several design cycles, in which the focus was on aesthetics, user interface, content and robustness (speed and reliability). The end-result is an easy-to-use application that runs on any device with a web-browser, in which the core ideas of a Council of virtual coaches were demonstrated with 92 end-users for a period of 5-9 weeks.The second main outcome is the project’s “Technical Demonstrator”, a software platform in which various state of the art components are integrated, allowing content creators to develop their own applications that employ multiple embodied conversational agents. Various end-user studies have been performed using the technical demonstrator, gathering evidence on e.g. user-interaction paradigms, and fundamental questions of vicarious persuasion. Released as a fully open-source platform under the name “Agents United”, the platform is fully available to the public through www.agents-united.org.For both demonstrators realistic sustainability plans have been developed and set in motion, focusing on commercial exploitation of the functional demonstrator, and a lively developers community around the technical demonstrator.Additional outcomes of the project include the WOOL Platform, the stand-alone open source software platform for authoring dialogues and integration in web- and mobile applications (www.woolplatform.eu). Additionally, the project will soon release an annotated video corpus of multi-coach-user interactions that was used as a tool to analyse interactions between multiple coaches and a user. Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far) The concept of the "Council of Coaches" is inherently new. A very strong focus from day 1 of the project has been on developing tangible prototypes to demonstrate this concept. Overall, the concept of Council of Coaches was successfully demonstrated by means of the functional demonstrator. Users showed good levels of engagement with the demonstrator.Furthermore, the project has realized an integrated, fully open-source software platform that allows content creators to develop applications supporting multiple AI-driven embodied conversational agents that constitutes a major step beyond the state of the art, both in terms of innovative aspects, as well as its fully open-source nature.In summary, Council of Coaches has pushed the state of the art in the following ways:- The novel idea of a group of virtual coaches for coaching on health and wellbeing was successfully demonstrated to provide a promising method of engaging end-users with their health.- The WOOL Platform was developed and released as open-source platform for authoring- and integrating scripted dialogues in web- and mobile applications. Where existing tools are often proprietary and never focus on the authoring process from a non-technical domain-expert’s point of view, WOOL is taking a place in the market for dialogue systems for serious applications (www.woolplatform.eu).- A methodology and vision on how to practically implement RRI principles in the daily workings of a RIA project has been established.- The coach-as-a-sensor concept has been demonstrated to measure data that is harder to measure through sensors, integrated through dialogues with the end user to capture e.g. emotional and cognitive data.- A holistic behavior framework was developed that takes data from different sources (mobile phones, off-the-shelf sensors, and the coaches themselves), and uses it to predict long-term behaviours.- An integrated dialogue system that advances the state of the art by combining expertise, models, and insights from the fields of argumentation and social conversation. The novelty of this aspect of the system lies in the way that various modules in the system work together to move from models of coaching topics and content, to structural models of dialog to determine the content of successive moves in the conversation, via models of social conversational intents that are needed to deliver a single piece of content, to actual delivery of the content in the form of agent behaviour.- A computational model to control the multimodal behaviours of virtual agents was developed based on studies to understand the relationship between task, social cohesion, and multimodal behaviours. A new corpus on group interactions has been developed in which video of a multi-party coaching sessions was analysed and annotated with dialogue and gesture moves. - A computational model of communicative gestures was also developed. While previous computational models work either with only beat gestures (which marks speech rhythm) or with only illustrative gestures (including iconic, metaphoric and deictic), we have developed a model which works for both types of gestures. Moreover, our model also considers eyebrow movements which are linked to speech prosody. Our model was trained on data which has been annotated with gesture types. The data was also segmented into gesture phases (preparation, stroke, retraction, hold). It allows our model to learn where to place a gesture stroke such that it aligns with the speech. couch-promo-banner-1.png
