Periodic Reporting for period 3 - SONORA (The Spatial Dynamics of Room Acoustics)
Periodo di rendicontazione: 2021-05-01 al 2022-10-31
The SONORA project aims to increase the general understanding of how complex sound scenes are impacted by the spatial dynamics of room acoustics. This knowledge is crucial in the design of signal processing algorithms for audio acquisition and reproduction problems in real-life situations, where moving sound sources and observers interact with room acoustics in a complicated manner.
A major part of the project will be devoted to the development of novel room acoustics models and to the unification of existing models. The room acoustics models developed in this project will be data-driven models with a physically motivated structure, and are expected to fill the existing gap between geometric and wave-based models. This will be achieved by formulating existing and novel models in a dictionary- based mathematical framework and introducing a new concept coined as the equivalent boundary model, aimed at relaxing the prior knowledge required on the physical room boundary.
A second part of the project will focus on the development of a protocol for measuring spatiotemporal sound fields. This protocol will be rooted in a novel sound field sampling theory which exploits the spatial sparsity of sound sources by invoking the compressed sensing paradigm.
Thirdly, novel signal processing algorithms capable of handling spatiotemporal sound fields will be designed. By employing recent advances in large-scale optimization and multidimensional scaling, fast and matrix-free algorithms will be obtained that do not require prior knowledge of the sound scene geometry.
The SONORA research results are anticipated to have a notable impact in various audio acquisition and reproduction problems, including acoustic signal enhancement, audio analysis, room inference, virtual acoustics, and spatial audio reproduction. These problems have many applications in speech, audio, and hearing technology, hence a significant benefit for industry and for technology end users is expected in the long run.
A major part of the project will be devoted to the development of novel room acoustics models and to the unification of existing models. The room acoustics models developed in this project will be data-driven models with a physically motivated structure, and are expected to fill the existing gap between geometric and wave-based models. This will be achieved by formulating existing and novel models in a dictionary- based mathematical framework and introducing a new concept coined as the equivalent boundary model, aimed at relaxing the prior knowledge required on the physical room boundary.
A second part of the project will focus on the development of a protocol for measuring spatiotemporal sound fields. This protocol will be rooted in a novel sound field sampling theory which exploits the spatial sparsity of sound sources by invoking the compressed sensing paradigm.
Thirdly, novel signal processing algorithms capable of handling spatiotemporal sound fields will be designed. By employing recent advances in large-scale optimization and multidimensional scaling, fast and matrix-free algorithms will be obtained that do not require prior knowledge of the sound scene geometry.
The SONORA research results are anticipated to have a notable impact in various audio acquisition and reproduction problems, including acoustic signal enhancement, audio analysis, room inference, virtual acoustics, and spatial audio reproduction. These problems have many applications in speech, audio, and hearing technology, hence a significant benefit for industry and for technology end users is expected in the long run.
Physics-driven models of acoustic wave propagation within a spatial domain have been developed by adopting a representation in which wave fields are approximated using series expansions around spatial sampling points chosen on or relative to the boundary of the domain. For domains with a cylindrical geometry, a novel convergence analysis and a practical algorithm to predict the optimal truncation number of the multipole expansion method (MEM) has been proposed. In domains having a geometry involving corner singularities, a modeling paradigm based on the method of fundamental solutions (MFS) has been established, where the sampling points are chosen relative to the corners of the domain. A highly efficient wave field simulation algorithm coined as the lightning wave equation solver has been developed, exhibiting beneficial features including controllable accuracy, low complexity, and suitability for time-domain simulation.
Data-driven models of acoustic wave propagation have been developed by considering compact representations of room impulse responses (RIRs) in a low-rank approximation framework. The relation between low-rank approximations and modal acoustic theory has been explored, and a fast convolution method using these compact RIR representations is under development.
A novel framework for solving inverse problems in room acoustics has been established, featuring a regularization method that allows to include a spatial sparsity prior in room inference problems and a suite of proximal gradient algorithms for numerically solving the resulting non-convex optimization problems.
Two heuristic methods that are widely employed in acoustic signal processing, i.e. the estimation of steered response power (SRP) maps and the deconvolution of exponential sine sweep (ESS) measurements, have been rigorously rederived in a Nyquist sampling framework, resulting in increased accuracy and/or reduced complexity.
Data-driven models of acoustic wave propagation have been developed by considering compact representations of room impulse responses (RIRs) in a low-rank approximation framework. The relation between low-rank approximations and modal acoustic theory has been explored, and a fast convolution method using these compact RIR representations is under development.
A novel framework for solving inverse problems in room acoustics has been established, featuring a regularization method that allows to include a spatial sparsity prior in room inference problems and a suite of proximal gradient algorithms for numerically solving the resulting non-convex optimization problems.
Two heuristic methods that are widely employed in acoustic signal processing, i.e. the estimation of steered response power (SRP) maps and the deconvolution of exponential sine sweep (ESS) measurements, have been rigorously rederived in a Nyquist sampling framework, resulting in increased accuracy and/or reduced complexity.
The SONORA project is anticipated to deliver a number of research results that go beyond the state of the art in room acoustics modeling, sensing, and processing, along the following themes:
(1) Time domain processing and convolution: dynamic sound scenes involving moving sources and observers call for a time-domain approach to modeling, sensing, and processing. This requirement conflicts with the widely used frequency-domain approach to wave propagation modeling and acoustic signal processing. The project’s heavy focus on efficiently representing and computing continuous-time convolutions is the key element to facilitating time-domain processing.
(2) Sampling and interpolation: by recognizing that all room acoustics models rely on a number of spatial reference points and that microphone array measurements provide spatiotemporal samples of an acoustic wave field, various modeling and measurement approaches can be unified in a sampling framework. The establishment of this framework moreover allows to develop optimal interpolation methods and rigorously analyze the impact of undersampling.
(3) Hybrid geometric and wave-based modeling: the project is expected to deliver new insights on how geometric models and wave-based models, that have traditionally been treated separately, are related. This will be achieved by interpreting wave field expansion terms as equivalent source signals, the position of which depends on the choice of the spatial reference points used in the expansion.
(1) Time domain processing and convolution: dynamic sound scenes involving moving sources and observers call for a time-domain approach to modeling, sensing, and processing. This requirement conflicts with the widely used frequency-domain approach to wave propagation modeling and acoustic signal processing. The project’s heavy focus on efficiently representing and computing continuous-time convolutions is the key element to facilitating time-domain processing.
(2) Sampling and interpolation: by recognizing that all room acoustics models rely on a number of spatial reference points and that microphone array measurements provide spatiotemporal samples of an acoustic wave field, various modeling and measurement approaches can be unified in a sampling framework. The establishment of this framework moreover allows to develop optimal interpolation methods and rigorously analyze the impact of undersampling.
(3) Hybrid geometric and wave-based modeling: the project is expected to deliver new insights on how geometric models and wave-based models, that have traditionally been treated separately, are related. This will be achieved by interpreting wave field expansion terms as equivalent source signals, the position of which depends on the choice of the spatial reference points used in the expansion.