Using FFT analysis and resynthesis to create infinite sustained sounds and spectral reverb.
One of my research interests is the modification of the sound characteristics of the classical guitar. This instrument features rich timbre but its notes have short sustain.
The use of Fast Fourier Transform (FFT) to analyze the sound of the instrument can allow the synthesis of sounds that have a similar pitch to that of the classic guitar, but which can be synthesized indefinitely.
A quick summary of objectives
In summary, the objectives of this final project are:
- The analysis of classical guitar sound in real time;
- Manipulation or processing performed in the frequency domain in order to generate either a sound of infinite sustain and a spectral delay;
- Sound resynthesis using the inverse FFT process (IFFT).
The project design
Although my research points to the possibility of achieving the goals of this project using either granular synthesis or delay lines (in a complex and possibly unstable configuration), the frequency domain sound manipulation allows me to further expand the functionality of the programmed patch, including pitch Control and time stretching.
For this project I opted for programming in Pure Data because this language allows me to execute the programmed patch on computers running operating systems based on Linux. This feature will allow me to use the patch on embedded systems based on Raspberry Pi, for example.
The patch contains 2 operation modes: a continuous analysis and resynthesis mode (spectral delay) and a manual triggering mode (freeze). The spectral delay mode will analyze blocks in a given time interval that can be set by the user and the output will respond according to the real time input intensity after N samples (spectral delay). The freeze mode will synthesize a continuous sound from the blocks acquired immediately after the user activates the function, i.e. push the freeze button.
The programming includes several concepts learned during the course MUMT-307 (Music \& Audio Computing II): Signal Analysis using STFT and resynthesis using IFFT, digital delay lines, feedback (in filters and frequency domain processing), windowing, FFT “hop” and “Phase”, overlapping, convolution, etc.
Particular issues that proved challenging / rewarding
Programming in Pure Data presents some extra challenges in relation to Max/MSP: The lack of objects in Pure Data makes it necessary to use multiple objects to perform functions that are easily solved with only one object in Max/MSP. An interesting example is the counter object. The Vanilla version of the PD does not have any counter object, being necessary to the use of the metro object and the creation of accumulators using mathematical operators with storage objects.
This problem can be solved by using the PD-Extended version or by using other distributions such as Pd-L2Ork (Purr Data). The use of PD-Extended is not the ideal solution since it is abandoned. Pd-L2Ork is the best option to use external and libraries that leave PD functionality similar to Max/MSP.
One last important comment about choosing which version of PD to use is the issue of compatibility: PD-Vanilla is still a good software choice if the intention is to make the programmed patch available in all operating systems and devices, including mobile phones.
All the references provided great information to build the patch. Some examples of different approaches for spectral freeze and spectral delay were found among Pure Data and Max/MSP community. Some of these patches provided good ideas and were implemented in the resulting patch, including visual feedback and “speed control” (Ralf’s Freeze3.pd) and practical tips for implementing procedures (Jean-francois Charles paper and patches in Max/MSP).
Unfortunately most implementations found in both the PD and Max/MSP communities have conceptual errors or present very simple implementations that do not provide sound quality or different configuration possibilities.
Even using the different procedures studied during the discipline, the resulting sound quality can vary drastically according to the sound source used and the signal-to-noise ratio. The patch proved itself moderately robust for sounds with concentrated energy in the medium-high regions of the spectra, such as trumpet’s high register or a whistle.
A mixing object has been programmed to control the relationship between the original sound and its synthesized version (depth control). This patch can be checked in the above figure (mixer).
For the low region of the spectrum (more present in classical guitar spectra) the noise level in the synthesized sound is considerably higher and greater attenuation of the more energetic frequency peaks is required. This attenuation is provided in the form of a noise filter control, which provides destructive interference to the signal to eliminate excess noise.
Example usage / demo
Appendix: FFT subpatch