Physical Modeling on FPGA

Detailed physical models of whole musical instruments – every string, plate, membrane and the enclosed air – need an enormous amount of computation. On normal computers they run many times slower than real time. Together with Florian Pfeifle we implemented such finite-difference models on Field Programmable Gate Arrays (FPGAs). An FPGA is a chip whose logic can be wired for one task, so thousands of model points are calculated in parallel at every time step. In this way, whole-geometry instrument models sound in real time and can be played by musicians, studied by researchers and changed by instrument makers while listening.

How the models are built

The instrument is split into its parts – strings, bridge, membrane or plates, air – each calculated in its own block on the chip. A routing layer couples the parts with each other, a control layer lets the user change parameters during playing, and an interface layer sends the sound to the computer or directly to an audio converter (PCIe, AC’97, I2S).

Model routing layer on the FPGA
The model routing layer: several instrument parts (geometries), each with its own control circuit, are coupled on the chip.

Banjo

The first complete instrument was the North American five-string banjo, with five strings, bridge, membrane, air cavity and a model of the plucking plectrum, running on a Xilinx Virtex-6 board. The model is played with a Leap Motion controller and sliders on the screen. Musicians can play it live; researchers can change the tension of the membrane at the tuning pegs; instrument makers can move the bridge on the membrane and hear the result immediately.

Plucked banjo notes from the FPGA model
Plucked banjo notes from the real-time model, with slipping noises of the plectrum when the playing position changes.
Spectrogram of banjo plucks with moving bridge
Several plucks of the same string while the bridge is moved on the membrane.

The real banjo, filmed with a high-speed camera: the string slipping off the finger pick and the bridge moving on the membrane. All videos are high-speed camera recordings, played back strongly slowed down.

Banjo string plucked with a finger pick.
Banjo bridge on the membrane with vibrating strings.

Violin

A real-time violin model consists of top and back plate, the enclosed air with its two f-hole openings, the bridge, four strings and a model of the bow–string interaction. It runs on an FPGA board connected to a PC and is played with a Leap Motion controller. As an example, the height of the arching of the top plate was increased step by step while bowing: the sound becomes brighter, but not in a simple linear way.

Violin model with increasing top plate height
Spectrogram (black and white) and spectral centroid (green) of the bowed violin model while the top plate height (dashed) is increased in steps of 0.05 mm.

Piano and signal processing

As an outlook, the method was extended to several larger Virtex-7 FPGAs, with the aim of simulating a whole grand piano in real time. FPGAs are also ideal for general audio signal processing with very low and stable latency, like real-time filtering. An introduction to FPGA hardware, hardware description languages, audio interfaces, filters and physical modeling is given in the chapter by Florian Pfeifle in the Springer Handbook of Systematic Musicology. Together with Math Verstraelen from the University of Twente we tested the WaveCore processor, a programmable many-core processor on FPGA built for acoustics, with a six-string instrument model, a plate model and a reverberation network.

See also Physical Modeling.

Literature

Pfeifle, F.: Real-Time Signal Processing on Field Programmable Gate Array Hardware. In: R. Bader (ed.): Springer Handbook of Systematic Musicology, 385-417, Springer 2018. https://doi.org/10.1007/978-3-662-55004-5_20

Pfeifle, F. & Bader, R.: Real-Time Finite-Difference Method Physical Modeling of Musical Instruments Using Field-Programmable Gate Array Hardware. J. Audio Eng. Soc. 63 (12), 1001-1016, 2015. https://doi.org/10.17743/jaes.2015.0089

Verstraelen, M., Pfeifle, F. & Bader, R.: Feasibility Analysis of Real-Time Physical Modeling Using WaveCore Processor Technology on FPGA. In: A. Mayer, V. Chatziioannou & W. Goebl (eds.): Proceedings of the Third Vienna Talk on Music Acoustics, 268-275, 2015.

Pfeifle, F. & Bader, R.: Real-Time Physical Modelling of a real Banjo geometry using FPGA hardware technology. In: R. Bader (ed.): Musical Acoustics, Neurocognition and Psychology of Music. Hamburger Jahrbuch für Musikwissenschaft 25, 71-86, 2009.