Publications

*co-first authorship

2024

ST-ITO: Controlling Audio Effects for Style Transfer with Inference-time Optimization
Christian J. Steinmetz, Shubhr Singh, Marco Comunità, Ilias Ibnyahya, Shanxin Yuan, Emmanouil Benetos, Joshua D. Reiss
International Society for Music Information Retrieval (ISMIR). November 2024
[ arXiv ] [ Demo ] [ Code ] ( Best Paper Award )

Diff-MST: Differentiable Mixing Style Transfer
Soumya Sai Vanka*, Christian J. Steinmetz*, Jean-Baptiste Rolland, Joshua Reiss, George Fazekas
International Society for Music Information Retrieval (ISMIR). November 2024
[ arXiv ] [ Demo ] [ Code ]

Modeling Analog Dynamic Range Compressors using Deep Learning and State-space Models

Hanzhi Yin, Gang Cheng, Christian J. Steinmetz, Ruibin Yuan, Richard M. Stern, Roger B. Dannenberg
Preprint. March 2024
[ arXiv ] [ Demo ]

ATGNN: Audio Tagging Graph Neural Network

Shubhr Singh, Christian J. Steinmetz, Emmanouil Benetos, Huy Phan, Dan Stowell
IEEE Signal Processing Letters. January 2024
[ arXiv ]

2023

High-Fidelity Noise Reduction with Differentiable Signal Processing

Christian J. Steinmetz, Thomas Walther Joshua D. Reiss
155th Convention of the Audio Engineering Society (AES). October 2023
[ arXiv ] [ Product ] [ Video ] [ Blog ]

General Purpose Audio Effect Removal

Matthew Rice*, Christian J. Steinmetz*, George Fazekas, Joshua D. Reiss
IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA). October 2023
[ arXiv ] [ Demo ] [ Code ] [ Video ] [ Dataset ] [ Demo ] ( Best Paper Award )

Modulation Extraction for LFO-Driven Audio Effects

Christopher Mitcheltree, Christian J. Steinmetz, Marco Comunità, Joshua D. Reiss
26th International Conference on Digital Audio Effects (DAFx). May 2023
[ arXiv ] [ Demo ] [ Code ] [ Video ]

Leveraging Neural Representations for Audio Manipulation

Scott H. Hawley and Christian J. Steinmetz
154th Convention of the Audio Engineering Society. May 2023
[ arXiv ] [ Demo ]

2022

Modelling black-box audio effects with time-varying feature modulation

Marco Comunità, Christian J. Steinmetz, Huy Phan, Joshua D. Reiss
Under Review. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP). November 2022
[ arXiv ] [ Demo ] [ Code ] [ Video ]

Style transfer of audio effects with differentiable signal processing

Christian J. Steinmetz, Nicholas J. Bryan, Joshua D. Reiss
To appear in the Journal of the Audio Engineering Society (JAES). September 2022
[ arXiv ] [ Demo ] [ Code ] [ Video ] [ Slides ]

Efficient neural networks for real-time modeling of analog dynamic range compression

Christian J. Steinmetz and Joshua D. Reiss
152nd Convention of the Audio Engineering Society. May 2022
[ arXiv ] [ Demo ] [ Code ] [ Video ] [ Slides ]

Direct design of biquad filter cascades with deep learning by sampling random polynomials

Joseph T. Colonel*, Christian J. Steinmetz*, Marcus Michelen, Joshua D. Reiss
IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP). May 2022
[ arXiv ] [ Code ] [ Demo ] [ Video ] [ Slides ]

HEAR 2021: Holistic Evaluation of Audio Representations

Joseph Turian, Jordie Shier, Humair Raj Khan, Bhiksha Raj, Björn W. Schuller, Christian J. Steinmetz, Colin Malloy, George Tzanetakis, Gissel Velarde, Kirk McNally, Max Henry, Nicolas Pinto, Camille Noufi, Christian Clough, Dorien Herremans, Eduardo Fonseca, Jesse Engel, Justin Salamon, Philippe Esling, Pranay Manocha, Shinji Watanabe, Zeyu Jin, Yonatan Bisk
Proceedings of Machine Learning Research (PMLR): NeurIPS 2021 Competition Track. March 2022
[ arXiv ] [ Code ] [ Web ] [ Talks ]

2021

Steerable discovery of neural audio effects

Christian J. Steinmetz and Joshua D. Reiss
5th Workshop on Machine Learning for Creativity and Design at NeurIPS. December 2021
[ arXiv ] [ Code ] [ Demo ] [ Video ] [ Slides ]

WaveBeat: End-to-end beat and downbeat tracking in the time domain

Christian J. Steinmetz and Joshua D. Reiss
151st Convention of the Audio Engineering Society. October 2021
[ arXiv ] [ Code ] [ Video ] [ Slides ]

Filtered noise shaping for time domain room impulse response estimation from reverberant speech

Christian J. Steinmetz, Vamsi Krishna Ithapu and Paul Calamia
IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. October 2021
[ arXiv ] [ Demo ] [ Video ] [ Slides ] ( Best Student Paper Award )

pyloudnorm: A simple yet flexible loudness meter in Python

Christian J. Steinmetz and Joshua D. Reiss
150th Convention of the Audio Engineering Society. May 2021
[ PDF ] [ Package ] [ Code ] [ Video ] [ Slides ] [ Blog ]

Automatic multitrack mixing with a differentiable mixing console of neural audio effects

Christian J. Steinmetz, Jordi Pons, Santiago Pascual, Joan Serrà
IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP). June 2021.
[ arXiv ] [ Code ] [ Demo ] [ Video ] [ Slides ]

2020

auraloss: Audio-focused loss functions in PyTorch

Christian J. Steinmetz and Joshua D. Reiss
Digital Music Research Network Workshop (DMRN+15). December 2020.
[ PDF ] [ Code ]

Randomized overdrive neural networks

Christian J. Steinmetz, Joshua D. Reiss
4th Workshop on Machine Learning for Creativity and Design at NeurIPS. December 2020.
[ arXiv ] [ Code ] [ Demo ] [ Talk ] [ Slides ] [ Blog ]

Theses

Learning to mix using neural audio effects in the waveform domain

Christian J. Steinmetz
MSc Thesis, Music Technology Group, Universitat Pompeu Fabra, Barcelona. September 2020.
[ Zenodo ]

Patents

System for automated multitrack mixing

Christian J. Steinmetz and Joan Serrà
Patent ES-P202030604 (Jun 22, 2020), EP-21731213.1 (Jun 16, 2021), JP-2022-578976 (Jun 16, 2021), US-18/012245 (Jun 16, 2021)
[ gpatents ]

Talks

Controlling Signal Processing Devices with Deep Learning

Invited talk @ Bose Research. January 2024
[ Slides ]

dasp: Differentiable Audio Signal Processors

Keynote @ SDX Workshop at ISMIR. November 2023
[ Web ] [ Slides ]

AI Town Hall by the AES Technical Comittee on Machine Learning and AI

Panelist @ 155th Convention of the Audio Engineering Society. October 2023
[ Web ]

AI Music Tools & Services: A status quo of what’s available right now

Panelist @ Reeperbahn Festival Conference. September 2023
[ Web ]

dasp: Differentiable Audio Signal Processors

Invited talk @ Neural DSP. May 2023
[ Slides ]

Controlling Audio Effects with Deep Learning

NYU MARL Seminar Series. March 2023
[ Slides ]

Style Transfer of Audio Effects with Differentiable Signal Processing

Stanford CCRMA DSP Seminar Series. November 2022
[ Slides ] [ Video ] [ Webpage ]

Audio Production Style Transfer with DeepAFx-ST

Harmonai Hangouts @ Stability AI. September 2022
[ Slides ] [ Video ]

Style Transfer of Audio Effects with Differentiable Signal Processing

Sonos Seminar Series. September 2022
[ Slides ]

Style Transfer of Audio Effects with Differentiable Signal Processing

Invited Talk @ Mitsubishi Electric Research Labs (MERL). September 2022
[ Slides ]

Style Transfer of Audio Effects with Differentiable Signal Processing

Music + AI Reading Group @ MILA. August 2022
[ Slides ] [ Video ]

Neural Audio Effects

The Audio Programmer (TAP) Meetup. July 2022
[ Slides ] [ Video ]

Differentiable Signal Processing for Audio Engineering

AI and Music Centre for Doctoral Training EAB Meeting. June 2022
[ Slides ]

Style Transfer of Audio Effects with Differentiable Signal Processing

Invited Talk @ Adobe Research. July 2022

AI for the Future of Music Production

Día de la Producción Musical @ Tecnológico de Monterrey. December 2021
[ Slides ]

Deep Learning for Automatic Mixing: Challenges and Next Steps

Music Demixing (MDX) Workshop at ISMIR. November 2021
[ Slides ] [ Video ] [ PDF ]

Artificial Intelligence, Machine Learning, & The Future of Music as We Know It!

Attack & Release Podcast. July 2021
[ Audio ] [ Notes ]

Deep Learning Approaches to Multitrack Mixing

AES Symposium: Applications of Machine Learning in Audio. November 2020
[ Slides ] [ Video ]

flowEQ: Using machine learning and MATLAB to build an intelligent EQ plugin

The Audio Programmer (TAP) Meetup. April 2020
[ Video ]

Applications of Machine Learning for Creative and Assistive Audio Plugins

Invited Talk @ Dolby Labs, Barcelona. November 2019
[ Slides ]

How Algorithms Help You Discover Your Next Song

Speaker at TEDx Clemson University. April 2019
[ Video ]

Tutorials

AI for multitrack music mixing

Soumya Sai Vanka, Christian J. Steinmetz, Gary Bromham, Marco A. Martínez Ramírez, Brecht De Man,
Workshop at 155th Audio Engineering Society Convention. October 2023
[ Web ] [ Code ] [ Slides ] [ Video ]

Deep learning for automatic mixing

Christian J. Steinmetz, Soumya Sai Vanka, Gary Bromham, Marco A. Martínez Ramírez
Tutorial at 23rd International Society for Music Information Retrieval Conference (ISMIR). December 2022
[ arXiv ] [ Web ] [ Code ] [ Slides ] [ Video ]

Audio Tech Talk Series

Department of Audio Engineering @ Clemson University. Spring 2019
[ Slides ]

Press

TechCrunch - Tape It’s software for musicians aims to deliver studio-quality noise reduction via AI
[ Article ]

Ableton Blog - AI and Music-Making: The State of Play
[ Article ]

Service

Conference Reviewer

International Conference on Machine Learning (ICML), 2023, 2024
Neural Information Processing Systems (NeurIPS), 2023
Neural Information Processing Systems (NeurIPS) Competition Track, 2022
IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2023
International Society for Music Information Retrieval (ISMIR) Conference, 2021, 2022, 2023
Convention of the Audio Engineering Society (AES), 152nd, 153rd, 154th, 155th, 156th
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023 (Outstanding Reviewer), 2024

Journal Reviewer

Journal of the Audio Engineering Society (JAES), 2022, 2023
EURASIP Journal on Audio, Speech, and Music Processing, 2023, 2024
IEEE Signal Processing Letters, 2022
Music & Science, 2021

Organizer

Neural Audio Plugin (NAP) Competition, 2023 [ web ]
Member, AES Technical Committee: Machine Learning and Artificial Intelligence , 2021 - Present
Member, HEAR 2021 NeurIPS Challenge: Holistic Evaluation of Audio Representations [ web ]

Awards

Best Paper Award at WASPAA 2023

General Purpose Audio Effect Removal. Fall 2023

Best Student Paper Award at WASPAA 2021

Filtered Noise Shaping for Time Domain Room Impulse Response Estimation From Reverberant Speech. Fall 2019

Gold Award at AES MATLAB Student Plugin Competition

flowEQ [ Webpage ] Fall 2019

Silver Award at AES MATLAB Student Plugin Competition

NeuralReveberator [ Webpage ] Fall 2018

KEMET Electrical and Computer Engineering Senior Design Award

Autonomous Air Hockey Robot [ Webpage ] Fall 2018

1st Place at IEEE SoutheastCon Student Paper Competition

Phase Analysis Plugin [ Webpage ] Spring 2018

1st Place at Clemson University IEEE Student Paper Competition

Phase Analysis Plugin [ Webpage ] Spring 2018

Finalist at AES Student Design Competition

Phase Analysis Plugin [ Webpage ] Fall 2017

Overall Student Media Contribution Award

WSBF Live Sessions [ Webpage ] Spring 2017

Hackathons

Judge at Timbre Tools Hackathon

February 2024 [ Webpage ]

Judge at CUhackit

Spring 2021

First Place at r/ProgrammerHumor Hackathon

findio [ Webpage ] Summer 2019

Domain.com Award at CUhackit

CUparkit [ Webpage ] Spring 2019

Mentor at CUhackit: Hello World

Spring 2019

Best Cloud Based Hack at CUhackit

auxCord [ Webpage ] Spring 2018

Participant at HackGT 4: New Heights

Recipe Revealer [ Webpage ] Fall 2018

Scholarships

UKRI Centre for Doctoral Training in Artificial Intelligence and Music (AIM) Studentship

Fall 2020 to Fall 2024

Audio Engineering Society Genelec Ilpo Martikainen Audio Visionary Scholar

Fall 2019 to Summer 2020

Audio Engineering Society Mary Lea Simpson Memorial Scholarship

Fall 2018 to Spring 2019

Charles C Kanapaux Scholarship

Fall 2015 to Spring 2019

Alan McCrary Johnstone Scholarship

Fall 2017 to Spring 2018

Cephus W Long Engineering Scholarship

Fall 2017 to Spring 2018

Carpenter Endowed Engineering Scholarship

Fall 2015 to Spring 2016

SC LIFE STEM Enhancement Scholarship

Fall 2015 to Spring 2018

SC LIFE Scholarship

Fall 2014 to Spring 2018

National Merit Corporate Scholarship

Fall 2014 to Fall 2018

Heritage Classic Foundation Scholarship

Fall 2014 to Spring 2018

Jeff Kanton Memorial Scholarship

Fall 2014

Richard R. Moore Scholarship

Fall 2014

Rotary Club of Hilton Head Island Scholarship

Fall 2014

Hilton Head Area Home Builders Association Scholarship

Fall 2014

Relevant Coursework

Math

Linear Algebra, Linear Programming, Ordinary Differential Equations, Time Series Analysis, Discrete Math Structures, Machine Learning I & II

Audio/Music

Acoustics of Music, Advanced Sound Reinforcement, History of Audio Technology, Audio Engineering I & II, Introduction to Digital Audio Effects, Computational Music Creativity, Music Information Retrieval, Music Perception and Cognition, Deep Learning for Audio and Music, Music Informatics, Music and Audio Programming, Audio Signal Processing for Music

Signal Processing

Random Signal Analysis, Signals and Systems, Digital Signal Processing, Communication Systems, Linear Control Systems