We do not use artificial intelligence as a black box that replaces listening, but as a tool to find order in large amounts of sound, and to connect the physics of musical instruments with perception and with the music of different cultures.

Self-organizing maps

Most of our work uses self-organizing maps (SOM, also called Kohonen maps). A SOM is a neural network that arranges sounds on a two-dimensional map according to their similarity – of timbre, rhythm, tonal system or performance. Similar pieces end up close together, different ones far apart. No categories are given beforehand, so the map shows which musical styles, instruments or players are related without our own expectations being built in. This is especially valuable for music from oral traditions, where Western categories often do not fit.

Physics-informed machine learning

Physical models of instruments can produce thousands of sounds for which every physical parameter – damping, tension, thickness, material – is known. Machine learning then finds out which parameters really change the sound and how we hear these changes. In turn, the physical parameters of a real instrument can be estimated from a recording alone. We use this for drum membranes, jazz drum heads, the piano soundboard and the Chinese hulusi.

Exhibitions

Our exhibition “Can you hear it? – Music and Artificial Intelligence” at the Museum für Kunst und Gewerbe Hamburg let visitors explore music with AI themselves. The results are presented at musicai.uni-hamburg.de. A further exhibition on Music and AI was shown in Yao’an, Yunnan, China, 2025. See also Exhibitions.

Beyond neural networks: the IPF

The Impulse Pattern Formulation (IPF) is a different kind of intelligent system: a compact nonlinear model that can play along with musicians, synchronise rhythms and even model social and political processes. In Social/Political Prediction IPF a self-organizing map is used to show the opinions of the participants.

Topics on this site that use AI

• Computational Phonogram Archiving (COMSAR) – similarity search in ethnomusicological archives
• Myanmar and China – timbre and tonal systems of Kachin and Uyghur music
• Indonesia / Bali – clustering of gamelan orchestras
• Percussion and Viscoelastic damping – machine learning of drum membranes and jazz drum heads
• Piano – timbre development of the piano analysed with machine learning
• Wind Instruments – machine learning of hulusi timbre
• IPF and Social/Political Prediction IPF
• Exhibitions

Literature

Bader, R., Alexandraki, C., Martínez, C. & Starakis, M.: Viscoelastic physical modeling synthesis and machine learning analysis of Jazz drum head tapering. J. Acoust. Soc. Am. 157 (4_Supplement), A107, 2025. doi

Martínez Orellanos, C. F. & Bader, R.: Analysis of nonlinear behavior of viscoelastic damping in musical membranes using physics-informed self-organizing maps. APL Mach. Learn. 3 (1), 016102, 2025. doi

Martínez Orellanos, C. F. & Bader, R.: Psychoacoustic analysis of viscoelastically damped membranes using physics-informed machine learning. Proceedings Forum Acusticum Euronoise 2025. pdf

Martínez Orellanos, C. F. & Bader, R.: Physics- and Psychoacoustics-Based Analysis of Viscoelastically Damped Membranes. Proceedings DAGA 2025. pdf

Linke, S., Wendt, G. & Bader, R.: Clustering of Indonesian and Western Gamelan Orchestras through Machine Learning of Performance Parameters. arXiv:2409.03713, 2024.

Bader, R., Linke, S. & Gernert, S.: Nonlinear dynamical social and political prediction algorithm for city planning and public participation using the impulse pattern formulation. Chaos 34, 093130, 2024.

Xia, Y. & Bader, R.: Machine Learning of Timbre Features for the Western Chinese Free-Reed Instrument Hulusi. Proceedings DAGA 2024, 1050-1053. pdf

Martínez, C. & Bader, R.: Machine Learning-Analysis of a Physics-informed Dataset of a Viscoelastic Damped Membrane. Proceedings DAGA 2024, 983-986.

Blass, M. & Bader, R.: Inhaltsbasiertes System zur Suche und Visualisierung von Musik in ethnomusikologischen Musikarchiven. In: R. Bader (Hrsg.): Computergestützte Archivierung von Tonträgern, 157-187, SpringerNature 2024.

Alexandraki, C., Starakis, M., Zervas, P. & Bader, R.: Inferring Drumhead Damping and Tuning from Sound Using Finite Difference Time Domain (FDTD) Models. Acoustics 5, 798–816, 2023. doi

Alexandraki, C., Starakis, M., Bader, R. & Zervas, P.: Machine learning of Finite-Difference Time Domain (FDTD) physical modelling sound simulation of drumhead paste pattern distributions. Proceedings Forum Acusticum 2023, 1-8. pdf

Plath, N. & Bader, R.: Piano Timbre Development Analysis using Machine Learning. arXiv preprint, 2021.

Bader, R., Blaß, M. & Franke, J.: Computational timbre and tonal system similarity analysis of the music of Northern Myanmar-based Kachin compared to Xinjiang-based Uyghur ethnic groups. arXiv:2103.08203, 2021.

Bader, R., Zielke, A. & Franke, J.: Timbre-based machine learning of clustering Chinese and Western Hip Hop music. Audio Engineering Society, Spring 2021. pdf

Blaß, M. & Bader, R.: Content Based Music Retrieval and Visualization System for Ethnomusicological Music Archives. In: R. Bader (ed.): Computational Phonogram Archiving. Springer Series ‘Current Research in Systematic Musicology’, Vol. 5, 145-174, 2019.

Bader, R. (ed.): Computational Phonogram Archiving. Springer Series ‘Current Research in Systematic Musicology’, Vol. 5, 2019.