FunDSP

Audio Processing and Synthesis Library for Rust

FunDSP is an audio DSP (digital dignal processing) library with a focus on usability.

It features a powerful inline graph notation that empowers users to accomplish diverse audio processing tasks with ease and elegance.

The custom notation taps into composable, zero-cost abstractions that express audio processing networks as Rust types.

FunDSP comes with a combinator environment containing a suite of audio components, math and utility functions and procedural generation tools.

This project is under construction! It is already useful for experimentation. However, some standard components are missing and breakage can be expected as we continue to experiment with best practices.

Uses

• Education
• Music making
• Sound hacking and audio golfing
• Audio synthesis for games and applications
• Prototyping of DSP algorithms

Graph Notation

FunDSP Composable Graph Notation expresses audio networks in an algebraic form, using graph operators. It was developed together with the functional environment to minimize the number of typed characters needed to accomplish common audio tasks.

Many common algorithms can be expressed in a natural form conducive to understanding, making FunDSP a superb platform for education. For example, a FM oscillator can be written simply as:

sine_hz(f) * f * m + f >> sine()

The above expression defines an audio graph that is compiled into a stack allocated, inlined form using the powerful generic abstractions built into Rust. Connectivity errors are detected at compile time, saving development time.

Audio DSP Becomes a First-Class Citizen

With no macros needed, the FunDSP graph notation integrates audio DSP tightly into the Rust programming language as a first-class citizen. Native Rust operator precedences work in harmony with the notation, minimizing the number of parentheses needed.

FunDSP graph expressions offer even more economy in being generic over channel arities, which are encoded at the type level. A mono network can be expressed as a stereo network simply by replacing its mono generators and filters with stereo ones, the graph notation remaining the same.

FunDSP Composable Graph Notation was developed by Sami Perttu, with contributions from Benjamin Saunders.

Principia

Component Systems

There are two parallel component systems: the static AudioNode and the dynamic AudioUnit.

Both systems operate on audio signals synchronously as an infinite stream.

The lower level AudioNodes can be lifted to block processing mode with the object safe AudioUnit interface via the AnUnit<X: AudioNode> wrapper. Block processing aims to maximize efficiency in dynamic situations.

AudioNodes can be stack allocated for the most part. Some nodes may use the heap for audio buffers and the like.

Sample Rate Independence

Of the signals flowing in graphs, some contain audio while others are controls of different kinds.

With control signals and parameters in general, we prefer to use natural units like Hz and seconds. It is useful to keep parameters independent of the sample rate, which we can then adjust as we like.

In addition to sample rate adjustments, natural units enable support for selective oversampling in nested sections that are easy to configure and modify.

Some low-level components ignore the sample rate by design, such as the single sample delay tick().

In both systems, a component A can be reinitialized with a new sample rate: A.reset(Some(sr)).

Audio Processing Environment

FunDSP prelude defines a convenient combinator environment for audio processing.

There are two name-level compatible versions of the prelude.

The default environment (fundsp::prelude) offers a generic interface. It is flexible and attempts to conform to Rust practices.

The hacker environment (fundsp::hacker) for audio hacking is fully 64-bit to minimize type annotations and maximize audio quality. The hacker interface uses 1 floating point type (f64) and 1 integer type (i64) only.

An application interfacing fundsp will likely pick the default environment for maximum flexibility, while experimenters will be drawn to the succinctness of the hacker prelude.

The aims of the environments are:

• Minimize the number of characters needed to type to express an idiom.
• Keep the syntax clean so that a subset of the hacker environment can be parsed straightforwardly as a high-level DSL for quick prototyping.
• Make the syntax usable even to people with no prior exposure to programming.
• Exemplify an effective procedural style.

Deterministic Pseudorandom Phase

FunDSP uses a deterministic pseudorandom phase system for generators. Generator phases are seeded from network structure and node location.

Thus, two identical networks sound identical separately but different when combined. This means that noise() | noise() is a stereo noise source, for example.

Operators

Custom operators are available for combining audio components inline. In order of precedence, from highest to lowest:

In the table, constant denotes an f32 or f64 value.

All operators are associative, except the left associative -.

Operators Diagram

Broadcasting

Arithmetic operators are applied to outputs channel-wise.

Arithmetic between two components never broadcasts channels: channel arities have to match always.

Direct arithmetic with f32 and f64 values, however, broadcasts to an arbitrary number of channels.

The negation operator broadcasts also: -A is equivalent with (0.0 - A).

For example, A * constant(2.0) and A >> mul(2.0) are equivalent and expect A to have one output. On the other hand, A * 2.0 works with any A, even sinks.

Fit

The fit (!) operator is syntactic sugar for chaining filters with similar connectivity.

It adjusts output arity to match input arity and passes through any missing outputs to the next node. The missing outputs are parameters to the filter.

For example, while lowpass() is a 2nd order lowpass filter, !lowpass() >> lowpass() is a steeper 4th order lowpass filter with identical connectivity.

Generators, Filters and Sinks

Components can be broadly classified into generators, filters and sinks. Generators have only outputs, while filters have both inputs and outputs.

Sinks are components with no outputs. Direct arithmetic on a sink translates to a no-op. In the prelude, sink() returns a mono sink.

Graph Combinators

Of special interest among operators are the four custom combinators: pipe ( >> ), bus ( & ), branch ( ^ ), and stack ( | ).

The pipe is a serial operator where components appear in processing order. Branch, stack, and arithmetic operators are parallel operators where components appear in channel order.

Bus is a commutative operator where components may appear in any order. The other operators are not commutative in general.

All four are fully associative, and each come with their own connectivity rules.

Pipe

The pipe ( >> ) operator builds traditional processing chains akin to composition of functions. In A >> B, each output of A is piped to a matching input of B, so the output arity of A must match the input arity of B.

It is possible to pipe a sink to a generator. This is similar to stacking. Processing works as normal and the sink processes its inputs before the generator is run.

Branch

Where the arithmetic operators are reducing in nature, the branch ( ^ ) operator splits a signal into parallel branches.

In A ^ B, both components receive the same input but their outputs are disjoint. Because the components receive the same input, the number of inputs in A and B must match. In A ^ B, the outputs of A appear first, followed with outputs of B.

Branching is useful for building banks of components such as filters.

Bus

The bus ( & ) operator can be thought of as an inline audio bus with a fixed set of input and output channels. It builds signal buses from components with identical connectivity.

In A & B, the same input is sent to both A and B, and their outputs are mixed together. Components in a bus may appear in any order.

The bus is especially useful because it does not alter connectivity: we can always bus together any set of matching components without touching the rest of the expression.

Both A + B and A & B are mixing operators. The difference between the two is that A + B is reducing: A and B have their own, disjoint inputs. In A & B, both components source from the same inputs, and the number of inputs must match.

Stack

The stack ( | ) operator builds composite components. It can be applied to any two components.

As a graph operator, the stack corresponds to the disjoint union. In A | B, the inputs and outputs of A and B are disjoint and they are processed independently, in parallel.

In stacks, components are written in channel order. In A | B | C, channels of A come first, followed by channels of B, then C.

Expressions Are Graphs

The expression A >> (B ^ C ^ D) defines a signal processing graph. It has whatever inputs A has, and outputs everything from B and C and D in parallel.

The whole structure is packed, monomorphized and inlined with the constituent nodes consumed. If you want to reuse components, define them as functions or closures, or clone them. See the prelude for examples of the former.

Connectivity is checked during compilation. Mismatched connectivity will result in a compilation error complaining about mismatched typenum types. The arrays Frame<T, Size> that connect components come from the generic-array and numeric-array crates.

Computational Structure

Graph combinators consume their arguments. This prevents cycles and imposes an overall tree shape on the resulting computation graph.

Implicit cycle prevention means that the built structures are always computationally efficient in the dataflow sense. All reuse of computed data takes place locally, inside combinators and components.

There are two main ways to structure the reuse of signals in FunDSP graph notation: branching and busing. Both are exposed as fundamental operators, guiding toward efficient structuring of computation. Dataflow concerns are thus explicated in the graph notation itself.

Input Modalities And Ranges

Some signals found flowing in audio networks.

Signal Flow Analysis

FunDSP features a comprehensive signal flow system that analyzes causal latencies and frequency responses in audio networks.

The system can calculate the frequency response of any linear network analytically by composing transfer functions and folding constants. Linear networks are constructed from filters, delays, and the operations of:

• Mixing.
• Chaining. Chaining of filters and delays maintains linearity.
• Constant scaling. Signals may be scaled by constant factors. Selected network inputs may also be marked constant for flow analysis.

Signal latencies are similarly analyzed from input to output in detail, facilitating automatic removal of pre-delay from effects chains.

For example, FIR filters can be composed inline from single sample delays (the tick opcode) and arithmetic. Signal flow analysis will readily reveal that a 2-point averaging filter has zero gain at the Nyquist frequency, while a 3-point averaging filter does not:

assert!((pass() & tick()).response(0, 22050.0).unwrap().norm() < 1.0e-9);
assert!((pass() & tick() & tick() >> tick()).response(0, 22050.0).unwrap().norm() > 0.1);

However, with appropriate scaling a 3-point FIR can vanish, too:

assert!((0.5 * pass() & tick() & 0.5 * tick() >> tick()).response(0, 22050.0).unwrap().norm() < 1.0e-9);

List of Filters

All filters in the list are linear. Verified frequency responses are available for all filters.

Free Functions

These free functions are available in the environment.

Component Opcodes

M, N, U are type-level integers. They are U0, U1, U2...

Subsampled Controls

envelope(f) is a node that samples a time varying control function f. For example, envelope(|t| exp(-t)) is an exponentially decaying envelope. A control function is something that is expected to change relatively slowly. Therefore, we can save time by not calling it at every sample.

The argument to the function is time in seconds. Whenever the node is reset, time resets to zero.

envelope is generic over channel arity: The return type of the function - scalar or tuple - determines the number of outputs.

The samples are spaced at an average of 2 ms apart, jittered by noise derived from pseudorandom phase. The values in between are linearly interpolated.

lfo (Low Frequency Oscillator) is another name for envelope.

Math And Utility Functions

Easing Functions

Easing functions are interpolation curves that remap the range 0...1. These math functions have the shape of an easing function.

Examples

For the practice of graph fu, some examples of graph expressions.

Examples From The Prelude

Many functions in the prelude itself are defined as graph expressions.

Equivalent Expressions

There are usually many ways to express a particular graph. The following expression pairs are identical.

Audio Graph Types

The AudioNode component system represents audio network structure at the type level.

The representation contains input and output arities, which are encoded as types U0, U1, ..., by the typenum crate. The associated types are AudioNode::Inputs and AudioNode::Outputs.

The representation contains sample and inner processing types when applicable, encoded in that order. These are chosen statically. The associated sample type, which is used to transport data between nodes, is AudioNode::Sample.

The encoding is straightforward. As an example, in the hacker prelude

noise() & constant(440.0) >> sine()

is represented as

An<BusNode<f64, NoiseNode<f64>, PipeNode<f64, ConstantNode<f64, U1>, SineNode<f64>>>>

The prelude employs the wrapper type An<X: AudioNode> containing operator overloads and other trait implementations.

License

MIT or Apache-2.0.

Future

• Overload division operator as an arithmetic operator once foundational overhaul is complete.
• Investigate whether adding more checking at compile time is possible by introducing opt-in signal units/modalities for AudioNode inputs and outputs. So if the user sends a constant marked Hz to an audio input, then that would fail at compile time.
• Examine and optimize performance.
• Implement conversion of graph to diagram (normalize operators to associative form). Layout and display a graph as a diagram and show the signals flowing in it. Allow user to poke at plug nodes while audio is playing.

TODO: Standard Components

• FIR filters.
• pluck
• Fractional delay line using an allpass filter (the standard delay should remain sample accurate only to not introduce extra processing).
• Variable delay and feedback lines.
• plug(tag). Mono parameter source. Tag is an arbitrary tag type. Tag can include metainformation about parameter. Add AudioNode interfaces: gather() and set(tag, value). The former returns all information about enclosed parameters and their current values.
• oversample(n, x). Oversample enclosed circuit x by n. Impose a default maximum sample rate to keep nested oversampling sensible.

TODO: Prelude

• melody(f, string): melody generator.