Replicating Audio Features: An Application of Machine Learning Concepts

Abstract

Researchers have found several applications for machine learning systems to extract, analyze, and replicate musical features. However, most approaches hinge on the reduction of a musical piece to its compositional aspects, rather than the raw audio of a musical recording (or any arbitrary audio recording). We sought to explore what applications machine learning concepts might have in music and audio analysis beyond compositional aspects. To exemplify this, a system was designed and implemented to leverage machine learning concepts and both extract and replicate empirical data about audio recordings’ timbres. To accomplish this, the “timbre” of an audio recording was defined by its frequency spectrum, i.e. the relative amplitudes of certain frequencies in the recording. The system is able to extract timbral data from a large number of sources, reduce the data to a useful core, and modify arbitrary audio recordings to replicate the timbre. From a qualitative standpoint, modified recordings generated by the system were convincing and encouraging, but from an empirical standpoint, there is significant room for improvement. However, this project proves that machine learning concepts can have usefulness beyond strictly musical analysis and in the realm of general audio analysis and modification.