C++ infer

[maozhiqiang]

hi @geneing ! I used the C++ code for infer! but the speed is so slowly
mels shape: (80, 500)
take times:53.759968996047974
Seven seconds of audio takes about 53 seconds！
my hparams

rnn_dims=256,
fc_dims=128,
sparsity_target=0.90,
How to optimize the params! thanks!

[geneing]

@maozhiqiang That's unexpectedly slow. On my computer (6 year old laptop) with the same hparams it runs a little slower than real time. Let's check a few things:

1. Did you run training long enough to prune the weights? Current code should print how pruning is progressing.
2. I'm assuming you ran convert_model to create a weight file.
3. Have you compiled the library with optimization? Without at least "-O2" optimization level the code will run very slowly. Surprisingly, "-O3" optimization produces a library that runs ~30% slower than with O2. I get best results with "-O2 -ffast-math"

Sorry about the lack of detailed instructions. I'll get it done...

[maozhiqiang]

hi @geneing ! Thank you for your reply. the training log is
epoch:1801, running loss:211.44811499118805, average loss:1.5547655514057945, current lr:0.0002390525520373693, num_pruned:530824 (0.9%)
and used convert_model to convert pth to .bin model
I used cmake for complied, how to used -O2 for complie
thank your

[geneing]

Run ccmake or cmake-gui . Switch to advanced mode ("t" in ccmake / a checkbox in cmake-gui). Find CMAKE_BUILD_TYPE entry and type RelWithDebInfo. Find CMAKE_CXX_FLAGS_RELWITHDEBINFO and edit to include -ffast-math flag.

You can also set build type from cmake command line: https://cmake.org/pipermail/cmake/2008-March/020347.html

[maozhiqiang]

I just add
SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -g -Wall -O2")
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -Wall -O2")
to CmakeList.txt and recomplied
and now speed
mels shape: (80, 500)
5.093661546707153
Seven seconds of audio takes about 5 seconds！
Is this correct? thank you ！

[geneing]

Sounds right. "Eigen3" library that I use, employs every templating trick to get best performance. When optimized, it's performance is excellent. In debug mode it is super inefficient.

A few more flags to play with "-ffast-math" "-march=native".

[maozhiqiang]

thank you @geneing ！ but The output is all noise, Is my place wrong！

[geneing]

1. Make sure that input mel is correct - e.g. use one of the training set inputs.
2. It may be easier to debug using python library

sys.path.insert(0,'lib/build-src-RelDebInfo') #use correct path to the shared library WaveRNNVocoder...so

import WaveRNNVocoder
mel = np.load(fname).T #check that the first dimension is 80. Plot to check that it looks like correct mel
wav = vocoder.melToWav(mel) #plot to see what wav looks like

[maozhiqiang]

thank you!

[maozhiqiang]

Hi @geneing !
I used synthesize.py , the outputs is normal,but using the test_wavernnvocoder.py has not worked!
sample :
test_0_mel.npy_orig.wav.zip
I don't know why?

[geneing]

@maozhiqiang Could you, please, attach the mel data you are using as an input. Then I can try to reproduce your problem.

[maozhiqiang]

@geneing thank you! test mels as follows
test_0_mel.npy.zip

[geneing]

@maozhiqiang Works for me with b2f5fc1.

test.zip

Commands:

import numpy as np
import librosa
import sys
sys.path.insert(0,'../WaveRNN-Pytorch/lib/build-src-RelDebInfo')

import WaveRNNVocoder
mel=np.load('eval/test_0_mel.npy')

vocoder=WaveRNNVocoder.Vocoder()
vocoder.loadWeights('../WaveRNN-Pytorch/model_outputs/model.bin')
wav=vocoder.melToWav(mel)
plot(wav)
librosa.output.write_wav('test.wav', wav, 16000)

The speech is a bit noisy and quiet. Cantonese?

[maozhiqiang]

@geneing thank you! my result is also noise, I saw the same code！ I don't know why!

[geneing]

I'm not sure how to help you. Here's the weight file I'm using.

model.bin.zip

[maozhiqiang]

thank you! I will try!