MUS 220a Fall 2006 - Final Project
CCRMA, Music Dept, Stanford University
12/11/2006
The original goal of this project was to simply create a body of interesting sounds with FM that would then be used by the custom controller MYRTLE. MYRTLE is an electronic controller that can simultaneously control many sounds via 3 ribbon strips and/or optical sensors. It also has the capability to adjust the sounds with effects sends and a fader. As the work progressed, I realized that it would be much more efficient and intuitive to create these sounds if there were some sort of interface for creating the sounds. Frequency Modulation can get very complicated very quickly, and keeping track of all the operators that affect one another can be a daunting organizational task In addition, I needed a way to allow the other members of the MYRTLE team to create their own FM sounds as well. Thus, the scope of the project was altered slightly to include the creation of a PD interface that facilitates the creation of FM synthesis easily and quickly.
Foundation:
To build an FM synthesis interface in PD required a working knowledge of the subtleties of FM, and for that I turned to Dr. John Chowning's book on the subject, "FM Theory and Applications." This book introduced the topic from the ground up by way of the 30 odd algorithms used in the DX7 synthesizer. Solidifying a conceptual and intuitive understanding of the technique was crucial for constructing the interface, and for creating interesting sounds.
Frequency Modulation is a sound synthesis technique using multiple signals that modulate together in predictable (and unpredictable!) ways. The signals are most commonly some type of oscillator, although non-periodic modulations offer some interesting timbral possibilities as well. A carrier frequency f(c) is multiplied by a modulation frequency f(m) which causes the frequency of f(c) to change periodically (at a rate equal to f(m)). The strength (amplitude) of the modulating frequency is called the modulation index (defined as the (frequency deviation)/f(m) ), and varying this affects the amount of frequency change applied to f(c). At low levels, this technique is analogous to vibrato, but as the modulation frequency becomes an audible frequency itself, the resulting modulation takes on a new pitch that is related but not equal to the two modulating signals. The timbre of this resultant sound (i.e. number and strength of sidebands in the sounds spectrum) is also related, and becomes more complicated as the modulation index increases, and as more and more operators are combined.
It is important to note that the modulation index is nothing more than the "volume" or gain of the modulating frequency. However, when this "volume" knob is cranked to eleven, interesting things begin to happen, since it is controlling the frequency deviation of the carrier, not its volume. As can be seen later, I took advantage of this relationship in configuring each operator, in the same way as was done with the DX7. More on this later...
Looking at this sounds created by a pair of modulated oscillators, it can be seen that the complexity of the sound increases very quickly, and with a very small computational cost. This was important because the sounds created were used with a live controller, where computer latency was a big factor. Keeping these patches "cheap" was of the utmost priority. FM is a very inexpensive synthesis method, as the spectrum of the sound created by only two oscillators can be quite complex. It is calculated as follows:
c +/- km
for k = 0,1,2,3...n
where n is approx = I + 2
for a more in-depth explanation of FM, please visit Sean Bratnober's project site:
http://ccrma-www.stanford.edu/~seanbrat/220a/final-proj.html
The operator is the basic unit of FM, used by Chowning in the DX7 and now fairly widespread. It consists of a carrier signal (the "raw pitch") that is optionally combined with an incoming modulating signal before being applied to an oscillator. The oscillator then takes that signal as it's frequency , either constant of varying . The oscillator is multiplied by the modulation index and then sent out to it's outlet.Eacho carrier signal can be multiplied by an integer that represents it's frequency ratio. Frequency ratios are very important in FM, because varying an operator's pitch by a ratio of the incoming raw pitch ensures that it's sonic relationship to the whole sound will remain unchanged as the incoming pitch varies.
In addition, each operator has inputs from an effects slider (mapped to a knob in MYRTLE) that scales the overall modulation index. There is a GUI for each operator that lets the user control all the FM parameters, including two I have not yet talked about:
Oscillator switching: There is nothing saying that every oscillator needs to be a simple cosine. Mapping the modulated signal to other generates can yield some very interesting results. Each operator can be switched between a cosine, phasor (sawtooth), white noise, and pink noise. The noise is limited in its usefulness, but adds some interesting flavor depending on how it is used.
ADSR:
So far FM has been described as having a constant output, but in fact much of the power of FM comes from amplitude enveloping known as ADSR. The Attack, Decay, Sustain, and Release of an operator can be set in the GUI. This comes in handy when you want a discrete, non-continuous FM sound. Furthermore, as it turns out, enveloping the volume of a modulator (f(m)) envelopes the spectrum of the carrier signal. What this means is that mapping a unique ADSR to each operator results in different parts of the FM instrument's spectrum attacking and decaying at different times. Among other things, this allows for the FM sound to evolve over time as you "bang" it. See below for examples of the difference between a constant tone and an ADSR tone.
The operator is built to be easily navigated, and entirely modular. Variable names are set at the time the operator is instantiated (via an argument), so that each operator can be controlled independently of one another. All parameters can be saved into a matrix that lives in the Instrument. The operator patch is fully commented to allow for easy hacking by anyone who happens along...
If the operator is the basic building block of FM, then the PD instruments are the things you build with them. An "instrument" in this scope, is simply a particular combination of operators with specific parameters. Each instrument can have an unlimited number of operators set in any arrangement (although practically, 6 are as many as you'll ever need). The way each operator carries or modulates the signal has everything to do with what sounds one can get out of the instrument. The DX7 synthesizer had 32 distinct algorithms at it's disposal, and each of those can be made simply by dragging each graphical operator to where you like it, and wiring it up with only one connection in and out. This modular structuring means that sounds can be created easily and quickly, with minimal time spent grappling with the internal logic of the patch.
In addition to the operator arrangement, each instrument has some very useful features in its GUI. A switcher allows the user to select constant sound, or ADSR triggered sound. The two outputs provide a very different perspective on any sound. It is trivial to set all the ADSR parameters for the operators and see how they sound immediately. A recall button immediately sets all the operators to the parameters saved by the user (remember, parameters control the sound and thus define the instrument).
The GUI allows for control of the instrument on the screen, but everything in the patch is also wired to be controlled externally by MYRTLE. The user can manually drag graphical sliders representing the incoming pitch data (raw pitch), and two effects. The first effect, as stated before, is mapped to modulation index scalar. The second effect is mapped to the length of the ADSR enveloping. Raising the slider stretches out the sound when you "bang" it. With the MYRTLE controller, ADSR is triggered and scaled by optical sensors that respond to a hand wave over the controller. MYRTLE also has buttons for transposing the pitch up and down by a Perfect Fifth, and there are GUI buttons to control that in the instrument as well.
Each instrument contains some additional synthesis objects, applied after the operators combine. The first is filtered noise, which can add a nice effect as the pitch changes (pitch is mapped to the center frequency and Q of the filter). In addition, a quick burst of noise can be triggered at the onset of an ADSR envelope, creating a more realistic attack for some sounds.
Instrument Scaling:
One of the characteristics of FM I became familiar with during this project was the pitch-sensitive nature of frequency modulation. Combining some number of operators leads to a complex timbre that depends on the frequency ratios of each operator. But as the incoming pitch becomes too high or too low, the relationships the led to an interesting timbre can break down a bit and new ones are introduced. This can be a feature instead of a problem, because instruments in the real world also have timbres that vary based on pitch. However, since the sounds I was interested in creating were rather specific, and because MYRTLE controls the pitch via three 12 inch strips, I decided to scale the raw pitch data to limit it to the range I desired for the instrument. It is similar to setting the length of a fret board or limiting the number of strings an instrument has.
I accomplished this scaling via table lookup. The incoming pitch data is used as an index to look up values in a table that become the pitch seen by the operators. In an effort to make the patch easy to use and highly configurable, I actually spent quite a bit of time constructing an automation tool that allows the user to create a lookup table with the pitch range they desire (the idea being that every new instrument that is created also has a preferred pitch range). This part of the patch can be found in the "input scaling" subpatch. Each table ramps up quickly from zero to the lowest desired pitch, and then ramps linearly to the highest desired pitch. This is done over an x axis of length 3640, a seemingly arbitrary number that in fact relates to the voltages received by the ribbon controllers on MYRTLE. This can be changed for non-MYRTLE use. A similar table lookup was used as a multiplier to the modulation indexes of each operator to avoid remnant oscillation when the pitch went to 0.
The only other scaling that is done is with the effects sends. MYRTLE sends data from 0-1023 that must be scaled differently for each instrument, since adjusting the overall mod index can have a HUGE effect on the sound, depending on the algorithm.
Input Scaling with the "Tablewritetool" abstraction:
Just like the individual operators that compose them, the instruments are designed to be easy to navigate, easy to change, and easy to combine. They are also built modularly, meaning that their variable names are set when the instrument is instantiated in another patch. There is a help file tucked away in a subpatch that offers some assistance to the FM or PD challenged. The working hierarchy for this project was that MYRTLE could switch between presets or combinations of several instruments that were made up of many operators.
A Simple implementation of several instruments:
Instrument Presets implemented in MYRTLE:
A view of the MYRTLE Controller PD interface:
Having succesfully delved into the world of FM synthesis, I would like to extend my knowledge by researching other forms of synthesis, and comparing their different methods and resultant sounds. One of FM's greatest strengths is it's low computation, but in today's world of computing, when processing speed and power is abundant, FM is merely a small part of the synthesis universe. There is much more to learn...
References:
Chowning, John and Bristow, David. "FM Theory & Applications By Musicians for Musicians" 1986 Yamaha Music Foundation