Wavenet

The WaveNet neural network architecture directly generates a raw audio waveform, showing excellent results in text-to-speech and general audio generation.

Moreover It can be used almost all sequence generation even text or image.

This repository provides some works related to WaveNet.

Features

• [x] Local conditioning
• [x] Generalized fast generation algorithm
• [x] Mixture of discretized logistics loss
• [ ] Parallel Wavenet

Generalized fast generation algorithm

We generalized Fast wavenet to filter width > 1 by using Multi-Queue structured matrix which has size of (Dilation x (filter_width - 1) x batch_size x channel_size).

When you generate a sample, you must feed the number of samples that have generated to the function. This is because the queue has to choose before queueing operation.

You can find easily the modified points and details of the algorithm in here.

Check the usage of the incremental generator in here.

Applications

Vocoder

Neural Vocoder can generate high quality raw speech samples conditioned on linguistic or acoustic features.

We tested our model followed @r9y9's works.

Audio samples are available at https://twidddj.github.io/docs/vocoder. See the issue for the result in here.

Pre-trained models

Getting Start

0. Download dataset

• the voice conversion dataset(for multi speaker, 16k): cmu_arctic
• the single speaker dataset(22.05k): LJSpeech-1.0

1. Preprocess data

python -m apps.vocoder.preprocess --num_workers 4 --name ljspeech --in_dir /your_path/LJSpeech-1.0 --out_dir /your_outpath/

2. Train model

python -m apps.vocoder.train --metadata_path {~/yourpath/train.txt} --data_path {~/yourpath/npy} --log_dir {~/log_dir_path}

3. Test model

You can find the codes for testing trained model in here.

Requirements

Code is tested on TensorFlow version 1.4 for Python 3.6.

References

• Wavenet
• Wavenet vocoder in Pytorch
• Wavenet vocoder in Tensorflow
• Fast wavenet

Related Papers

• "WaveNet: A Generative Model for Raw Audio"
• "Fast Wavenet Generation Algorithm"
• "PixelCNN++"
• "Parallel WaveNet: Fast High-Fidelity Speech Synthesis"