Second Language Tutoring using Social Robots

Project Information

L2TOR

Grant agreement ID: 688014

DOI

10.3030/688014

Project closed

EC signature date 12 November 2015

Start date 1 January 2016

End date 31 December 2018

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 3 042 565,00

EU contribution

€ 3 042 562,00

Coordinated by

UNIVERSITY OF PLYMOUTH
United Kingdom

Periodic Reporting for period 2 - L2TOR (Second Language Tutoring using Social Robots)

Reporting period: 2017-07-01 to 2018-12-31

Summary of the context and overall objectives of the project

The L2TOR (pronounced ‘el tutor’) project aims to design a child-friendly tutor robot that can be used to support teaching preschool children a second language (L2) by interacting with children in their social and referential world. The evaluation of the tutor robot will focus on teaching English as L2 to native speakers of Dutch, German and Turkish, and teaching Dutch and German as L2 to immigrant children speaking Turkish as a native language. The robot’s social behaviour will be based on how human tutors interact with children, and will not only use verbal communication, but also non-verbal communication, such as gestures and other forms of communication. The robot will be able to adaptively respond to children’s actions and engage with them in tutoring interactions. The child will be provided with increasingly complex stimuli and utterances in the second language, as well as appropriate feedback that supports the child’s language development.

The four main project objectives are:

1. Study the science and technology of social robots for second language tutoring with children in a preschool setting. The outcome of this objective will be the demonstration of an embodied digital learning environment, consisting of a tutor robot and supporting technologies, that can interact with children in an engaging manner to support children’s second language learning. This system will be adaptive such that it can be personalized for children at specific levels of linguistic competency, thus optimising the effectiveness of language tutoring.

2. Define the pedagogy of robot assisted language tutoring. The project will extend pedagogical guidelines for robot assisted language tutoring: from the behaviour of the robot to how it should be utilised by preschool teachers. The L2TOR robot will be evaluated by assessing the effectiveness of the tutor robot with respect to particular learning targets in different domains (number, space and narrative).

3. Determine design principles of developing a social robot for (second) language tutoring with a clear commercial focus. The project will produce open-access document that specifies a “cookbook” for industrial stakeholders on how to design a robot assisted tutoring system with commercial potential.

4. Innovate multimodal interaction management for robotic tutors. The project will design and evaluate three prototypes which will be tested in real-world settings in schools in Germany, The Netherlands, Turkey and the UK. In addition, industrial demonstrators will be on offer to show at public and commercial events.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

1. Development of a software stack to support language tutoring using a social robot and a touchscreen; the software consists of eight components which coordinate perception, planning, output generation, data logging and intra-process communication. The main focus of the software system is to support a large-scale scientific evaluation of the contribution social robots can make to second language tutoring.

2. Rigid evaluation of social signal processing techniques for Human-Robot Interaction, including speech recognition, voice activity detection, face detection and recognition. Various technologies have been evaluated and tested with an eye on their performance in the context of Human-Robot Interaction with young children.

3. Observations of second language learning and tutoring with the aim to inform the design of the robot tutor. Tutoring sessions between teachers and children have been recorded and annotated to get a view on how human tutors approach lecturing and tutoring of second language. Specific attention as paid to feedback, the use of L1 and L2 during tutoring, and scaffolding of the material.

4. A range of experiments and pilot studies which explore various parameters impacting on L2 learning created valuable insights into how to design a tutoring robot. For example, we studied how the robot’s behaviour impacts on the children’s learning performance (too friendly a robot has a negative impact, probably caused by children experiencing cognitive overload; but a more “verbally available” robot does not make a difference). We studied if children benefit from learning L2 spatial relationships with real objects, rather than with objects shown on a screen (no difference in performance was observed, probably because the spatial concepts are established already). Or we studied if personalised tutoring methods, using Bayesian Knowledge Tracing, made a difference.

5. A first pilot study has been completed, testing all technical components and the tutoring scripts, which provided valuable feedback in preparation for the large-scale study planned for 2018.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Already the L2TOR consortium is the leading team in studying second language tutoring; there is a second initiative running in the US, comprised of the MIT, Yale and USC, which runs a similar research programme. L2TOR's research and innovation sets itself apart from other initiatives through its domain (second language tutoring) and the scope of the studies we are running (long-term studies with hundreds of children).

There is a large amount of interest for the concept of using social robots as tutoring aids. While the teaching profession is justifiably cautious of yet another classroom technology, there seems to be growing interest in the potential offered by personalised tutoring by social robots. It is easy to imagine a future where each classroom would benefit from having one or several robots helping children on a one-to-one basis. We are at this moment exploring this further through our outreach activities and public events.

Child learning a second language with a robot tutor

Child learning a second language with a robot tutor

Child learning a second language with a robot tutor

Child learning a second language with a robot tutor

Group of children learning from a robot tutor

Child learning a second language with a robot tutor with experimenter present