Enhancing Motion Interaction through Music Performance

Informations projet

MIM

N° de convention de subvention: 659232

DOI

10.3030/659232

Projet clôturé

Date de signature de la CE 9 Avril 2015

Date de début 4 Janvier 2016

Date de fin 3 Janvier 2019

Financé au titre de

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Coût total

€ 232 160,40

Contribution de l’UE

€ 232 160,40

Coordonné par

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE CNRS
France

Periodic Reporting for period 2 - MIM (Enhancing Motion Interaction through Music Performance)

Période du rapport: 2018-01-04 au 2019-01-03

Résumé du contexte et des objectifs généraux du projet

As Information Communications Technology (ICT) is now spreading outside of workspaces towards our everyday life, new elements of human life such as culture, emotion or experience must be embraced in technology for Human-Computer Interaction (HCI). Part of this new objective in HCI is to build technologies that are “closer”, or more “natural”, to human, promoting the use of body movement in HCI. Exploring this challenge has been the cornerstone of digital arts and especially performing arts involving digital media. For example, music technologists have leveraged on motion sensing technology to explore new types of interaction between the body and digital medias. Nevertheless, there is a general agreement that these new interactive systems remain less satisfying in terms of expressive control compared to traditional instruments, and, importantly, do not allow to acquire skills through them.

The general objective of the project MIM is to investigate motor learning mechanisms in expert motor activity, especially instrumental music practice, in order to enhance movement computational models in movement-based musical interaction systems. The project MIM addresses three specific scientific objectives:
- Objective 1: Study the role of movement task variation in music skill acquisition,
- Objective 2: Study the nature of movement variability and learning schedules in the development of motor learning,
- Objective 3: Build interactive systems facilitating motor learning.

Travail effectué depuis le début du projet jusqu’à la fin de la période considérée dans le rapport et principaux résultats atteints jusqu’à présent

The project started by an exhaustive literature review on sensorimotor learning and music performance.

A first experimental study has been conducted showing that the motor task variability has a dissociable effect on timing and motor skill acquisition in piano sequence learning. The results were published in PLoS ONE:
- Caramiaux, B., Bevilacqua, F., Wanderley, M., and Palmer, C. Dissociable effects of practice variability on learning motor and timing skills. PLOS ONE. 13(3), 1-18. 2018.

A second experimental study has been initiated (pilot study has been run) investigating the individual-specificity of motor variability in piano sequence learning and the role of cognitive and biomechanical processes on motor variability. A conference proceedings showing preliminary results has been published in MOCO:
- Caramiaux, B., Bevilacqua, F., Palmer, C., and Wanderley, M. Individuality in Piano Performance Depends on Skill Learning. Proceedings of the 4th International Conference on Movement Computing. 2017

Design methodology to transfer this knowledge to the realisation of movement-based music systems have been explored. A collaborative paper reports on design highlights:
- Höök, K., Caramiaux, B., Erkut, C., Forlizzi, J., Hajinejad, N., et al. Embracing First-Person Perspectives in Soma-Based Design. Informatics. 5(1), 8. 2018

New design methodologies have been applied to better understand complex motor learning and learning schedule:
- Rivière, JP, Fdili Alaoui, S, Caramiaux, B., and Mackay WE. How Do Dancers Learn to Dance? A first-person perspective of dance acquisition by expert contemporary dancers. Proceedings of the 5th International Conference on Movement and Computing, 6. 2018

Progrès au-delà de l’état des connaissances et impact potentiel prévu (y compris l’impact socio-économique et les conséquences sociétales plus larges du projet jusqu’à présent)

While often considered as noise stemming from the motor or the sensory system, motor variability has recently been shown to have an important role in sensorimotor learning. Underlying processes vary the motor output in order to learn faster and better certain changes in environment dynamics (motor adaptation) or a new control policy (skill acquisition). This new perspective of motor variability has gained interest since the seminal Nature article in 2014. In music performance, variability has been barely explored. Few studies have shown that motor variability can be used to highlight chunking. However, motor variability as a parameter for skill acquisition in music remains unexplored.

Our first experimental study brings a new contribution to the state of the art in music performance and sensorimotor learning showing the impact of motor variability on motor and timing skill learning.

Based on our first experiments, we extended the methodology to include learner-centred persepctive of learning. These qualitative data helped the understanding of complex motor learning and drove the design of an interactive prototype aiming at documenting learning schedule and facilitating the learner along the process.

The impact is important for music pedagogy and more generally body-based activity, with implications in the design of innovative skill-facilitating interactive designs. These systems aim to provide adapted learning schedules to the learner. Such impact has recently been discussed during the Piano & gesture capture workshop at GREAM, at University of Strasbourg, and during a workshop dedicated to the topic at the Conference NIME (New Interfaces for Musical Expression) in Copenhagen, and finally a wider-scoped research workshop on Human-Machine Collaboration in Embodied Interaction, at IRCAM, Paris.

Piano skill acquisition