Enhancing Motion Interaction through Music Performance

Projektinformationen

MIM

ID Finanzhilfevereinbarung: 659232

DOI

10.3030/659232

Projekt abgeschlossen

EK-Unterschriftsdatum 9 April 2015

Startdatum 4 Januar 2016

Enddatum 3 Januar 2019

Finanziert unter

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Gesamtkosten

€ 232 160,40

EU-Beitrag

€ 232 160,40

Koordiniert durch

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE CNRS
France

Periodic Reporting for period 2 - MIM (Enhancing Motion Interaction through Music Performance)

Berichtszeitraum: 2018-01-04 bis 2019-01-03

Zusammenfassung vom Kontext und den Gesamtzielen des Projekts

As Information Communications Technology (ICT) is now spreading outside of workspaces towards our everyday life, new elements of human life such as culture, emotion or experience must be embraced in technology for Human-Computer Interaction (HCI). Part of this new objective in HCI is to build technologies that are “closer”, or more “natural”, to human, promoting the use of body movement in HCI. Exploring this challenge has been the cornerstone of digital arts and especially performing arts involving digital media. For example, music technologists have leveraged on motion sensing technology to explore new types of interaction between the body and digital medias. Nevertheless, there is a general agreement that these new interactive systems remain less satisfying in terms of expressive control compared to traditional instruments, and, importantly, do not allow to acquire skills through them.

The general objective of the project MIM is to investigate motor learning mechanisms in expert motor activity, especially instrumental music practice, in order to enhance movement computational models in movement-based musical interaction systems. The project MIM addresses three specific scientific objectives:
- Objective 1: Study the role of movement task variation in music skill acquisition,
- Objective 2: Study the nature of movement variability and learning schedules in the development of motor learning,
- Objective 3: Build interactive systems facilitating motor learning.

Arbeit, die ab Beginn des Projekts bis zum Ende des durch den Bericht erfassten Berichtszeitraums geleistet wurde, und die wichtigsten bis dahin erzielten Ergebnisse

The project started by an exhaustive literature review on sensorimotor learning and music performance.

A first experimental study has been conducted showing that the motor task variability has a dissociable effect on timing and motor skill acquisition in piano sequence learning. The results were published in PLoS ONE:
- Caramiaux, B., Bevilacqua, F., Wanderley, M., and Palmer, C. Dissociable effects of practice variability on learning motor and timing skills. PLOS ONE. 13(3), 1-18. 2018.

A second experimental study has been initiated (pilot study has been run) investigating the individual-specificity of motor variability in piano sequence learning and the role of cognitive and biomechanical processes on motor variability. A conference proceedings showing preliminary results has been published in MOCO:
- Caramiaux, B., Bevilacqua, F., Palmer, C., and Wanderley, M. Individuality in Piano Performance Depends on Skill Learning. Proceedings of the 4th International Conference on Movement Computing. 2017

Design methodology to transfer this knowledge to the realisation of movement-based music systems have been explored. A collaborative paper reports on design highlights:
- Höök, K., Caramiaux, B., Erkut, C., Forlizzi, J., Hajinejad, N., et al. Embracing First-Person Perspectives in Soma-Based Design. Informatics. 5(1), 8. 2018

New design methodologies have been applied to better understand complex motor learning and learning schedule:
- Rivière, JP, Fdili Alaoui, S, Caramiaux, B., and Mackay WE. How Do Dancers Learn to Dance? A first-person perspective of dance acquisition by expert contemporary dancers. Proceedings of the 5th International Conference on Movement and Computing, 6. 2018

Fortschritte, die über den aktuellen Stand der Technik hinausgehen und voraussichtliche potenzielle Auswirkungen (einschließlich der bis dato erzielten sozioökonomischen Auswirkungen und weiter gefassten gesellschaftlichen Auswirkungen des Projekts)

While often considered as noise stemming from the motor or the sensory system, motor variability has recently been shown to have an important role in sensorimotor learning. Underlying processes vary the motor output in order to learn faster and better certain changes in environment dynamics (motor adaptation) or a new control policy (skill acquisition). This new perspective of motor variability has gained interest since the seminal Nature article in 2014. In music performance, variability has been barely explored. Few studies have shown that motor variability can be used to highlight chunking. However, motor variability as a parameter for skill acquisition in music remains unexplored.

Our first experimental study brings a new contribution to the state of the art in music performance and sensorimotor learning showing the impact of motor variability on motor and timing skill learning.

Based on our first experiments, we extended the methodology to include learner-centred persepctive of learning. These qualitative data helped the understanding of complex motor learning and drove the design of an interactive prototype aiming at documenting learning schedule and facilitating the learner along the process.

The impact is important for music pedagogy and more generally body-based activity, with implications in the design of innovative skill-facilitating interactive designs. These systems aim to provide adapted learning schedules to the learner. Such impact has recently been discussed during the Piano & gesture capture workshop at GREAM, at University of Strasbourg, and during a workshop dedicated to the topic at the Conference NIME (New Interfaces for Musical Expression) in Copenhagen, and finally a wider-scoped research workshop on Human-Machine Collaboration in Embodied Interaction, at IRCAM, Paris.

Piano skill acquisition