Combining music with AI to facilitate a leap in audio technology

Dr Martin Parker and his group from the Reid School of Music at the University of Edinburgh are making significant strides at the intersection of music and technology. This dynamic team has been working closely with DataMind Audio as part of Dr Parker's project “Machining Sonic Identities”. 

The group is transforming how music is created and experienced through a suite of innovative audio plugins, one of which is based around the Real-time Audio Variational Encoder (RAVE), developed at the Institute for Research and Coordination in Acoustics/Music (IRCAM), Paris. Their work uses variational auto-encoder technology that functions like sonic alchemy, enabling musicians to reshape sounds in dynamic and imaginative new ways. 

The challenge 

In simple terms, the magic of this technology lies in its ability to transform sound from a live musical instrument into the sound of a different instrument – think of playing a sequence on a flute and hearing it back as if played by a cello, in (almost) real-time. The journey to develop this technology presented many challenges. The team sought to move beyond using neural networks to produce sound, and towards creating musically useful outputs that could be integrated “out of the box”. This is a nuanced endeavour that requires sophisticated, iterative model training and fine-tuning. Dr Parker’s team initially struggled with the pioneering nature of the project, the steep learning curve in handling raw audio data, and a lack of access to reliable GPUs. 

Our solution 

Recognising the unique needs of DataMind Audio, EPCC provided access to advanced computational resources via the Edinburgh International Data Facility (EIDF), including powerful GPUs from NVIDIA's H100 and H200 series. EPCC also provided DataMind Audio with advice and support from its EIDF support team. Dr Parker’s team noted that, without this support, they would still be dealing with the fundamental challenges they encountered previously when using a different cloud-based service.

EPCC was also instrumental in refining the DataMind Audio technical workflow. DataMind Audio worked with the EIDF support team to establish efficient protocols for accessing and managing large datasets, and to set up seamless file transfers between various third-party storage solutions. Both teams also built tools to ensure the efficient deployment of models, thereby making the DataMind Audio workflow significantly more efficient.  

Impact and future prospects 

The collaboration between the Reid School of Music, DataMind Audio and EPCC goes beyond technological innovation; it also enriches culture. The project has already facilitated new artistic expressions through performances that utilise models interacting with sounds from musical instruments that museum visitors typically cannot touch, let alone play. A new, reactive sculpture will open at St Cecilia’s Hall in October 2025 where new sounds can be synthesised from the models made of various untouchable instruments in St Cecilia’s collection.

DataMind Audio has also demonstrated an ethical use of creative works in an AI context by building models in close collaboration with artists. Fifty per cent of sales from their artists' "brain shop" go directly to artists, and models are crafted carefully with the artists themselves and are not published without permission. 

As this partnership continues to grow, the next 12 months are expected to be transformative. Dr Parker, DataMind Audio and EPCC are eager to share their experiences and innovations with a wider audience, helping to further explore what is possible in digital music production. 

This collaboration illustrates a successful model, demonstrating how specialised computing resources and expertise can empower creative industries to explore new frontiers and overcome traditional developmental limitations. Importantly the whole team has artistic and creative training yet find themselves working around the cutting edge of contemporary computing. This combination of artistic practice and deep computation offers an important partnership that others may follow in future.  

“Machining Sonic Identities” is an Arts and Humanities Research Council-funded Bridging Responsible AI Divides (BRAID) project.

Contact us

To explore how EPCC's expertise can support your business goals, please contact our Commercial Manager Julien Sindt.