The Spatial Dynamics of Room Acoustics

Project Information

SONORA

Grant agreement ID: 773268

DOI

10.3030/773268

Project closed

EC signature date 7 March 2018

Start date 1 May 2018

End date 31 October 2023

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 1 999 825,00

EU contribution

€ 1 999 825,00

Coordinated by

KATHOLIEKE UNIVERSITEIT LEUVEN
Belgium

Periodic Reporting for period 4 - SONORA (The Spatial Dynamics of Room Acoustics)

Reporting period: 2022-11-01 to 2023-10-31

Summary of the context and overall objectives of the project

The SONORA project aims to increase the general understanding of how complex sound scenes are impacted by the spatial dynamics of room acoustics. This knowledge is crucial in the design of signal processing algorithms for audio acquisition and reproduction problems in real-life situations, where moving sound sources and observers interact with room acoustics in a complicated manner.
A major part of the project will be devoted to the development of novel room acoustics models and to the unification of existing models. The room acoustics models developed in this project will be data-driven models with a physically motivated structure, and are expected to fill the existing gap between geometric and wave-based models. This will be achieved by formulating existing and novel models in a dictionary- based mathematical framework and introducing a new concept coined as the equivalent boundary model, aimed at relaxing the prior knowledge required on the physical room boundary.
A second part of the project will focus on the development of a protocol for measuring spatiotemporal sound fields. This protocol will be rooted in a novel sound field sampling theory which exploits the spatial sparsity of sound sources by invoking the compressed sensing paradigm.
Thirdly, novel signal processing algorithms capable of handling spatiotemporal sound fields will be designed. By employing recent advances in large-scale optimization and multidimensional scaling, fast and matrix-free algorithms will be obtained that do not require prior knowledge of the sound scene geometry.
The SONORA research results are anticipated to have a notable impact in various audio acquisition and reproduction problems, including acoustic signal enhancement, audio analysis, room inference, virtual acoustics, and spatial audio reproduction. These problems have many applications in speech, audio, and hearing technology, hence a significant benefit for industry and for technology end users is expected in the long run.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The main results of the SONORA project are summarized here, along the three overall objectives: modeling, sensing, processing.

Modeling: we have worked to develop novel room acoustics models that are capable of capturing the spatiotemporal behavior of complex sound scenes. These models possess desirable properties for signal processing purposes by being scalable to varying room types and sizes, robust to physical uncertainties and variations, physically interpretable, perceptually relevant, invertible and cheaply auralizable.
- A novel equivalent source model for room acoustics, which associates a directional impulse responses rather than a free-field Green's function to each equivalent source:
https://doi.org/10.1109/TASLP.2017.2730284
- A novel data-driven room acoustics model based on low-rank matrix/tensor decompositions, which comes with a solid physical justification that is rooted in the modal theory of room acoustics:
https://eurasip.org/Proceedings/Eusipco/Eusipco2021/pdfs/0000111.pdf
- A novel convergence theorem for modeling acoustic wave propagation with the multipole expansion method (MEM), that allows to optimally select the MEM truncation order:
https://doi.org/10.1137/20M1370914
- A novel relation between geometric and wave-based models for acoustic wave propagation, based on the observation that the solution to the boundary integration equation can be asymptotically represented by only considering specular reflection points on the boundary of the domain:
https://appfa2023.silsystem.solutions/atti/001143.pdf

Sensing: we have aimed to develop a novel framework for sensing room acoustics, embracing novel sampling theories and practical measurement protocols for spatiotemporal sound fields.
- A classical Nyquist-Shannon sampling and interpolation theory for a number of spatiotemporal sound field sensing problems in which a rigorous foundation was previously lacking:
https://arxiv.org/abs/2012.09499
https://doi.org/10.1109/I3DA48870.2021.9610902
https://eurasip.org/Proceedings/Eusipco/Eusipco2022/pdfs/0002256.pdf
- A compressed sensing framework for acoustic source localization and power-spectral-density (PSD) estimation, both in single-sensor and multi-sensor contexts:
https://doi.org/10.1186/s13636-023-00304-8
https://arxiv.org/abs/2306.08514
https://doi.org/10.1109/WASPAA58266.2023.10248095
- Optimal microphone selection/positioning algorithms for sensing spatiotemporal sound fields in various problem settings:
https://doi.org/10.1109/IWAENC53105.2022.9914798
https://tinyurl.com/3xa5fz68
- An extensive open-source database of source/receiver signals, RIR measurements, and noise field measurements with diverse microphone arrays and room types:
https://doi.org/10.1186/s13636-023-00284-9

Processing: we have designed novel signal processing algorithms addressing basic estimation, prediction, and simulation problems related to audio acquisition and reproduction in complex sound scenes, leveraging the desirable properties of the newly developed room acoustics models.
- A collection of numerical optimization algorithms that are particularly suited for estimating equivalent source models for room acoustics:
https://arxiv.org/abs/1803.01621
- A low-rank-promoting algorithm for estimating low-rank data-driven models for room acoustics directly from source/receiver data:
https://doi.org/10.1109/TASLP.2023.3240650
- Novel single- and multi-channel convolution algorithms that can operate directly on low-rank room acoustics models, resulting in fast convolution at low latency:
https://ftp.esat.kuleuven.be/stadius/mjalmby/23-149.pdf
https://ftp.esat.kuleuven.be/stadius/mjalmby/23-150.pdf
- New algorithms targeted at solving the fundamental problem of sorting and clustering TDOA estimates corresponding to first-order and second-order room reflections:
https://eurasip.org/Proceedings/Eusipco/Eusipco2021/pdfs/0001730.pdf
https://doi.org/10.1109/ICASSP49357.2023.10096005

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The SONORA project has delivered a number of research results that go beyond the state of the art in room acoustics modeling, sensing, and processing, along the following themes:
(1) Time domain processing and convolution: dynamic sound scenes involving moving sources and observers call for a time-domain approach to modeling, sensing, and processing. This requirement conflicts with the widely used frequency-domain approach to wave propagation modeling and acoustic signal processing. The project’s heavy focus on efficiently representing and computing continuous-time convolutions is the key element to facilitating time-domain processing.
(2) Sampling and interpolation: by recognizing that all room acoustics models rely on a number of spatial reference points and that microphone array measurements provide spatiotemporal samples of an acoustic wave field, various modeling and measurement approaches can be unified in a sampling framework. The establishment of this framework moreover allows to develop optimal interpolation methods and rigorously analyze the impact of undersampling.
(3) Hybrid geometric and wave-based modeling: the project has delivered new insights on how geometric models and wave-based models, that have traditionally been treated separately, are related. This has been achieved by showing under which conditions the (wave-based) solution to the boundary integral equation admits a geometric interpretation.

presentation of AES Board of Governors Award in Milan, May 2018

interactive auralization experiment with singers at Alamire Interactive Lab

visit of Her Majesty the Queen of Belgium to the Alamire Interactive Lab, Feb 2020

spatial audio measurement setup at Alamire Interactive Lab

1684501697833.jpeg

spatial audio measurement setup at Nassau Chapel

spatial audio measurement setup at SONORA lab

project meeting at the Alamire Analytic Lab

lecture presentation at Audio Engineering Society Symposium in Ghent, May 2019

spatial audio measurement setup at Alamire Interactive Lab

spatial audio measurement setup at Nassau Chapel