zakaton / Pink Trombone
Programming Languages
Projects that are alternatives of or similar to Pink Trombone
Sound is generated in the glottis (at the bottom left), then filtered by the shape of the vocal tract. The voicebox controls the pitch and intensity of the initial sound - Neil Thapen
๐ฃ๏ธ Pink Trombone - Bare-handed Speech Synthesis
A programmable version of Neil Thapen's famous and wonderful Pink Trombone
๐ Table of Contents
๐ Enabling and Disabling the UI
๐บ Manipulating Vocal Tract Constrictions
๐ฆ Setting Up
-
Save a local copy of
pink-trombone.min.jsandpink-trombone-worklet-processor.min.jsand make sure they're both in the same relative location (the first will import the other as a Audio Worklet Processor) -
In your HTML
<head></head>element, insert the file in a script element as a module:
<script src="pink-trombone.min.js" type="module"></script>
- In your HTML
<body></body>element, insert the following custom element:
<pink-trombone></pink-trombone>
- In your JavaScript code, grab the
<pink-trombone></pink-trombone>element:
var pinkTromboneElement = document.querySelector("pink-trombone");
- Add a
loadeventListener to thepinkTromboneElementelement:
pinkTromboneElement.addEventListener("load", myCallback);
- In the
"load"callback, assign an Audio Context using.setAudioContext(myAudioContext)(if none is specified, an Audio Context instance is created for you):
function myCallback(event) {
pinkTromboneElement.setAudioContext(myAudioContext)
}
This method returns a Promise once the AudioWorkletProcessor module is loaded.
- In the promise resolution, a Pink Trombone audio node is created, which you can connect to other audio nodes from the scope of the
<pink-trombone></pink-trombone>element:
function myCallback(event) {
pinkTromboneElement.setAudioContext(myAudioContext)
.then(() => {
const audioContext = pinkTromboneElement.audioContext
pinkTromboneElement.connect(audioContext.destination);
});
}
๐ Producing Sound
๐ To start generating sound, run the .start() method:
pinkTromboneElement.start();
๐ค To stop generating sound, run the .stop() method:
pinkTromboneElement.stop();
๐ Enabling and Disabling the UI
๐ To show the interactive visualization:
pinkTromboneElement.enableUI();
โ๏ธ To start animating the visualization:
pinkTromboneElement.startUI();
๐ To stop animating the visualization:
pinkTromboneElement.stopUI();
๐ To hide the interactive visualization:
pinkTromboneElement.disableUI();
๐๏ธ Audio Parameters
The audio parameters of the Pink Trombone audio node can be accessed from the <pink-trombone></pink-trombone> element's scope:
๐๏ธ Intensity
pinkTromboneElement.intensity;
๐ต Frequency
pinkTromboneElement.frequency;
๐ Tenseness
pinkTromboneElement.tenseness;
๐ข Loudness
pinkTromboneElement.loudness;
ใฐ๏ธ Vibrato
pinkTromboneElement.vibrato.frequency;
pinkTromboneElement.vibrato.gain;
pinkTromboneElement.vibrato.wobble;
๐ Tongue
// 'index' and 'diameter' refer to the tongue's location in the mouth
pinkTromboneElement.tongue.index;
pinkTromboneElement.tongue.diameter;
To change the voiceness between voiced and voiceless, change the .tenseness and .loudness audio parameters as follows:
function setVoiceness(voiceness) {
const tenseness = 1 - Math.cos((voiceness) * Math.PI * 0.5);
const loudness = Math.pow(tenseness, 0.25);
pinkTromboneElement.tenseness.value = tenseness;
pinkTromboneElement.loudness.value = loudness;
}
// voiced
setVoiceness(1);
// voiceless
setVoiceness(0);
Later on I may add a .voiceness audio parameter that automates this - for now I'm just adopting the original version
๐บ Manipulating Vocal Tract Constrictions
Vocal Tract constrictions comprise of an object containing .index and .diameter Audio Parameter properties that are implicitly connected to the Pink Trombone audio node
To add a vocal tract constriction:
var myConstriction = pinkTromboneElement.newConstriction(indexValue, diameterValue);
To set a vocal tract constriction:
myConstriction.index.value = newIndexValue;
myConstriction.diameter.value = newDiameterValue;
To remove a vocal tract constriction:
pinkTromboneElement.removeConstriction(myConstriction);
๐ Common Phonemes
For reference, here are some preset index & diameter preset values for some phonemes:
๐ Tongue phonemes:
รฆ [pat]
- index : 14.93
- diameter : 2.78
ษ [part]
- index : 2.3
- diameter : 12.75
ษ [pot]
- index : 12
- diameter : 2.05
ษ [port (rounded)]
- index : 17.7
- diameter : 2.05
ษช [pit]
- index : 26.11
- diameter : 2.87
i [peat]
- index : 27.2
- diameter : 2.2
e [pet]
- index : 19.4
- diameter : 3.43
ส [put]
- index : 17.8
- diameter : 2.46
u [poot (rounded)]
- index : 22.8
- diameter : 2.05
ษ [pert]
- index : 20.7
- diameter : 2.8
๐บ Vocal Tract Constriction phonemes: voiced and voiceless consonants share the same values, differing in voiceness
-
-
(ส, ส) ["s" in "pleasure"]
- index : 31
- diameter : 0.6
-
(z, s) ["z" in "zoo"]
- index : 36
- diameter : 0.6
-
(v, f) ["v" in "very"]
- index : 41
- diameter : 0.5
-
-
-
(g, k) ["g" in "go"]
- index : 20
- diameter : 0
-
(d, t) ["d" in "den"]
- index : 36
- diameter : 0
-
(b, p) ["b" in "bad"]
- index : 41
- diameter : 0
-
-
-
(ล) ["ng" in "hang"]
- index : 20
- diameter : -1
-
(n) ["n" in "not"]
- index : 36
- diameter : -1
-
(m) ["m" in "woman"]
- index : 41
- diameter : -1
-
๐ Developer Showcase
Send us an email at [email protected] if you have a cool application made with our api!
๐ Developer Wishlist
Our time is limited, so we'd greatly appreciate it if you guys could implement some of these ideas:
- [ ] IPA Speak n' See ๐ฃ๏ธ๐ฌ - Take input speech from the user using the Media Recording API and approximate their articulation using the Pink Trombone, allowing speakers to visualize how they speak.
- [ ] Phonetic Voice Editor ๐น๐โจ๏ธ - Create a cross between a Text Editor and a Digita Audio Workstation, where the user can type in phonemes instead of characters, with automation to programmatically adjust the cadence, pitch, and other features over time.
- [ ] SSML Simulator ๐๐ฌ - Implement a Speech Synthesis Markup Language emulator that can take an utterance and process the speech request using Pink Trombone's audio processing
๐ Bibliography
- Julius O. Smith III, "Physical audio signal processing for virtual musical instruments and audio effects."
- Story, Brad H. "A parametric model of the vocal tract area function for vowel and consonant simulation." The Journal of the Acoustical Society of America 117.5 (2005): 3231-3254.
- Lu, Hui-Ling, and J. O. Smith. "Glottal source modeling for singing voice synthesis." Proceedings of the 2000 International Computer Music Conference. 2000.
- Mullen, Jack. Physical modelling of the vocal tract with the 2D digital waveguide mesh. PhD thesis, University of York, 2006.