Enhancing Audio Transformation through the Integration of Machine Learning and Digital Signal Processing Techniques

Machine learning techniques have demonstrated significant potential in various audio processing applications. However, the current trend in AI research favours larger, more computationally intensive "black box" models. While these models may achieve impressive performance, they often lack the transparency and tweak-ability necessary to align with the specific artistic vision and requirements of musicians. To be truly useful in music production, audio transformation tools need to be responsive, highly customisable, and capable of real-time performance.

The objective of this PhD project is to explore the integration of classical digital signal processing (DSP) techniques into machine learning frameworks to create fast, modular and creative audio transformation models. By leveraging the explicit use of musical theoretical structures employed in traditional DSP algorithms with the expanded generative modelling capabilities of machine learning, this research aims to achieve two primary goals:
Improving the robustness, controllability and transparency of machine learning-based audio transformation models
Expanding the feature set and capabilities of DSP algorithms while maintaining their lightweight and real-time performance benefits.

The project aims to accomplish these objectives through several key tasks. Firstly, it involves transforming traditional DSP algorithms into parametrically driven tools for audio generation and transformation. Secondly, the project seeks to design lightweight machine learning models that, by leveraging the music theoretical foundation inherent in these algorithms, reduce computational costs during training and enhance the models' resilience to small training sets. Lastly, the project aims to implement techniques that enhance the inference efficiency of these models, enabling fast performance even on basic computers with limited compute resources.

By bridging the gap between classical DSP techniques and machine learning frameworks for audio processing, this project seeks to develop robust and transparent audio transformation models. By incorporating the musical theoretical structures used in traditional DSP algorithms while harnessing the power of machine learning, a toolkit of novel musical audio transformations and generators will be created. This toolkit will offer musicians enhanced flexibility, responsiveness, and real-time performance, ultimately advancing audio processing technology in the music production domain.