CPU only? #1

quantumlump · 2024-11-04T17:05:10Z

It seems to be running in CPU only, even though I used this prompt to start it in Docker:

docker run -it --rm --gpus all -p 7860:7860 athomasson2/ebook2audiobookpiper-tts:latest

Am I doing something wrong? Looks like it is possible based on this thread:
rhasspy/wyoming-piper#5

Thanks!

ps. While I found that xtts has better phrasing, the hallucinations (made up words, vocal artifacts) are worse than Piper which make the narration harder to understand unfortunately. I tried messing with a lot of the advanced settings (temperature etc. but couldn't get xtts more consistent than the base settings).

I am excited for Piper finetuning btw! Is there a way to be notified when finetuning is available? Also, I have made some high quality xxts finetunes, I would be happy to send them to you to attach to the project if you want. Just let me know!

DrewThomasson · 2024-11-04T17:45:40Z

About the piper-tts GPU use

hm, not gona lie, I just never looked at this repo in a while XD, it's defiantly possible to have gpu work
I'll look at adding that --cuda thing when I find time :) 👍

About fine-tuning piper tts

you should be able to fine tune pipertts , although its much harder to lol

Docs on it

https://github.com/rhasspy/piper/blob/master/TRAINING.md

Youtube Video on it

https://www.youtube.com/watch?v=67QvWOp3dkU

You say Hallucinations are the issue?
I might have a better repo that doesn't hallucinate lol
(Only limitation is that styletts2 only works for English)
ebook2audiobookStyletts2

https://github.com/DrewThomasson/ebook2audiobookSTYLETTS2

It's pretty close to xtts with easy voice cloning, just less emotional, but runs a lot faster
I forgot if I gave it GPU functionality tho lol but it should auto-detect gpu

DrewThomasson · 2024-11-04T17:49:10Z

ALSO You say you have some high quality xtts fine-tunes? 👀

Slap em on hugginface, send me links to them I'll check them out ! 😄

I'm planning on eventually getting a database of known xtts fine-tuned models to add to the ebook2audiobookxtts project 😄

DrewThomasson · 2024-11-04T18:22:56Z

I can confirm the fine-tuning piper-tts colabs still work as of this moment lol

I've used them before to fine tune a piper-tts model on Death from puss and boots 😅

https://huggingface.co/drewThomasson/piper_tts_finetune_death_from_puss_and_boots

Be aware it's very labor intensive for fine-tuning piper-tts 😭

They made XTTS fine-tuning so much easier 😢

quantumlump · 2024-11-04T18:23:36Z

Perfect! Here is the first one. I'll get the rest to you later:
https://huggingface.co/jdana/Bob_Newfound

Styletts2 was worse than Piper and xtts from my very limited experiments I just did

I will have to try out that piper finetune tutorial, thanks for sharing! With the tech moving so fast, I wonder if it is worth it to dive into that now or just wait until someone makes an easy webui 😂

quantumlump · 2024-11-04T18:28:50Z

Wait how do I find the link to the Piper-tts Colab?

DrewThomasson · 2024-11-04T18:32:21Z

Noted lol

I'm uploading the newest docker image for piper tts that supports GPU! :)

school wifi is slow 😅

DrewThomasson · 2024-11-04T18:34:08Z

Wait how do I find the link to the Piper-tts Colab?

The piper-tts Notebooks should be found here? 🤔
https://github.com/rhasspy/piper/tree/master/notebooks

quantumlump · 2024-11-04T18:40:15Z

Noted lol

I'm uploading the newest docker image for piper tts that supports GPU! :)

school wifi is slow 😅

Hell yeah! Thanks so much, and that was crazy fast, I hope you are prioritizing your studies btw 😟

Wait how do I find the link to the Piper-tts Colab?

The piper-tts Notebooks should be found here? 🤔 https://github.com/rhasspy/piper/tree/master/notebooks

I thought you meant that there was a piper training workflow in a Google Colab 😅 I got excited lol

DrewThomasson · 2024-11-04T18:45:24Z

Yeah the notebooks are `for opening in google colab`

If you click on any of them you should see a button for "open in google colab" lol

DrewThomasson · 2024-11-04T18:47:45Z

Here cause school wifi is too slow

DOCKER PIPER-TTS GPU

docker run -it --gpus all -p 7860:7860 --platform=linux/amd64 registry.hf.space/drewthomasson-ebook2audiobookpiper-tts-gpu:latest python app.py

You can also duplicate the hugginface space if you want to give it a gpu from huggingface at like 40 cents an hour lol

https://huggingface.co/spaces/drewThomasson/ebook2audiobookpiper-tts-GPU

quantumlump · 2024-11-04T18:52:15Z

Yeah notebooks for google colab lol, If you click on any of them you should see a button for "open in google colab" lol

Woah! Great, thank you!

Here cause school wifi is too slow

DOCKER PIPER-TTS GPU
docker run -it --gpus all -p 7860:7860 --platform=linux/amd64 registry.hf.space/drewthomasson-ebook2audiobookpiper-tts-gpu:latest python app.py
You can also duplicate the hugginface space if you want to give it a gpu from huggingface at like 40 cents an hour lol

https://huggingface.co/spaces/drewThomasson/ebook2audiobookpiper-tts-GPU

🔥🔥🔥🔥🔥🔥🔥🔥🔥

quantumlump · 2024-11-04T20:03:07Z

Hmm, still having trouble getting Piper to work with GPU

Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
Model loaded successfully.

DrewThomasson · 2024-11-04T20:44:14Z

Do you have a Cuda capable Nvida GPU? cause it only works with Nvidia GPU's

What output do you get when you run this?

docker run --gpus all nvidia/cuda:11.0-base nvidia-smi

DrewThomasson · 2024-11-04T21:04:13Z

Here, I tried fixing it by adding

nvidia-cuda-toolkit
libcudnn8
libcudnn8-dev
libegl1
libopengl0
libxcb-cursor0

into the packages.txt for the huggingface space

Try again with this comand to make it force pull the latest docker image and see if this works ✨

docker run -it --gpus all -p 7860:7860 --platform=linux/amd64 --pull always registry.hf.space/drewthomasson-ebook2audiobookpiper-tts-gpu:latest python app.py

DrewThomasson · 2024-11-04T21:08:37Z

I also created a free google colab of it with GPU support! :)

Confirmed to work in my testing
Pretty fast, aorund 1-2 seconds per sentence running on GPU :)

quantumlump · 2024-11-04T21:43:41Z

Hm, I tried the new pull and still not working, also the Colab says notebook not found

I am on Windows 10, RTX 3090

docker run --gpus all nvidia/cuda:12.3.1-base-ubuntu22.04 nvidia-smi
The prompt above runs, not this one that you sent me:
docker run --gpus all nvidia/cuda:11.0-base nvidia-smi

Does it have to be v11?

If it is confirmed to work on Colab it must be something on my end that I can troubleshoot.

DrewThomasson · 2024-11-04T21:46:23Z

OP you right, here try this for the google colab now I gave the wrong link

https://colab.research.google.com/github/DrewThomasson/ebook2audiobookpiper-tts/blob/main/notebooks/ebook2audiobook_piper_tts.ipynb"

Or this:

quantumlump · 2024-11-04T21:53:11Z

Were you able to confirm in the Colab that it was using the GPU, not the CPU?

I don't have Colab Pro so I can't check

DrewThomasson · 2024-11-04T21:58:22Z

Yes I can confirm that the Google colab runs on `GPU` and gives you `FREE GPUs`

You don't need Colab Pro to use the GPUS of google colab

You can see here on the free version

quantumlump · 2024-11-04T21:58:58Z

lol I do but I wanted to check when the program was running if it was utilizing the CPU or the GPU, like task manager etc.

Pretty sure you need to do something like this to see live if the GPU is being used?

quantumlump · 2024-11-04T22:00:42Z

The updated pull runs fine on my computer btw, not sure that wasn't clear. But it still uses the CPU instead of my 3090

DrewThomasson · 2024-11-04T22:07:13Z

wait I think your right >:|

I'll try another method and see what happens

DrewThomasson · 2024-11-04T22:33:50Z

Welp finally got it to work,

I'll try swapping out a command or something

DrewThomasson · 2024-11-04T22:34:42Z

I got that result running this test script in google colab

!pip install piper-tts
!pip install onnxruntime-gpu


import subprocess
import time

start_time = time.time()

subprocess.run(
    "echo 'Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!' | piper --model en_US-lessac-medium --output_file welcome.wav --cuda True",
    shell=True
)

end_time = time.time()
duration = end_time - start_time
print(f"GPU CUDA Command took {duration:.2f} seconds to run.")


import subprocess
import time

start_time = time.time()

subprocess.run(
    "echo 'Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!' | piper --model en_US-lessac-medium --output_file welcome.wav",
    shell=True
)

end_time = time.time()
duration = end_time - start_time
print(f"CPU Command took {duration:.2f} seconds to run.")

DrewThomasson · 2024-11-05T00:38:17Z

Nevermind it appears to be broken lol

The files produced by the GPU were actually empty

So >:|

DrewThomasson · 2024-11-05T00:43:49Z

I'm putting this on hold,

As it doesn't appear to be possible

Hit me up if you get piper itself running on your computer tho utilizing the GPU

Cause I can't figure it out for the life of me

Wouldn't even have to be modifying my code all I'd need is some example of any line of code that runs on GPU :(

quantumlump · 2024-11-05T02:02:09Z

lol thanks for trying, I will give it a go and see if I can make any headway. CPU speeds are workable, I am just curious to see how much faster it is using GPU 😄

DrewThomasson · 2024-11-05T02:03:35Z

Same lol

I can't Seem to find any definitive answer of it online for how much faster it is :/

DrewThomasson · 2024-11-05T18:59:58Z

Well update: i found this space

looks like it might be able to use the gpu?

https://huggingface.co/spaces/Gregniuki/Pipertts

When I Duplicated the space on gpu it seemed to work for me hmmm but GPU didn't seem that much faster . . . which is odd

you could attempt at running it locally as a docker perhaps?

here is my gpu duplicate version that should work on your computer as a docker
https://huggingface.co/spaces/drewThomasson/Pipertts

docker run -it -p 7860:7860 --platform=linux/amd64 \
	-e HF_TOKEN="YOUR_VALUE_HERE" \
	registry.hf.space/drewthomasson-pipertts:latest

DrewThomasson · 2024-11-05T19:00:31Z

Do tell if you see it utilizing your GPU tho if you test it out :)

quantumlump · 2024-11-05T23:08:32Z

Well update: i found this space

looks like it might be able to use the gpu?

https://huggingface.co/spaces/Gregniuki/Pipertts

When I Duplicated the space on gpu it seemed to work for me hmmm but GPU didn't seem that much faster . . . which is odd

you could attempt at running it locally as a docker perhaps?

here is my gpu duplicate version that should work on your computer as a docker https://huggingface.co/spaces/drewThomasson/Pipertts
docker run -it -p 7860:7860 --platform=linux/amd64 \
	-e HF_TOKEN="YOUR_VALUE_HERE" \
	registry.hf.space/drewthomasson-pipertts:latest 

Hm, can't get the webui to open

I tried pretty hard to change some files around with the original ebook2audiobook package but kept running into Docker build problems when installing 😑

Maybe I will try a little more, I feel like I am close.

Would be interesting if it ends up that the CPU version is faster for some reason 😶

DrewThomasson · 2024-11-06T17:23:38Z

Yeah doesn't seem like the space utilizes the GPU Nevermind

:/

quantumlump · 2024-11-06T21:10:52Z

I tried a bunch of stuff and couldn't get it to work.. sigh.. ur guys' job is hard

I will move on to trying out Piper finetunes!

DrewThomasson · 2024-11-07T00:30:40Z

https://huggingface.co/spaces/Gregniuki/Pipertts/discussions/1#672c0972b28c400174cb6e43

Ok so getting somewhere 👀

DrewThomasson · 2024-11-07T03:27:10Z

Ok SO WE HAVE An ANSWER ON SPEED INCREASE

From the guy:

On 1050 to was 5x speed of hugging spaces CPU especially for long 10 minutes and longer texts

quantumlump · 2024-11-07T04:25:06Z

Nice! I tried adapting his app code, changing the onnxruntime-gpu version, torch etc. for my 3090 to no avail. lol it feels so doable but I haven't had any luck. Curious to see what kind of performance he was getting in his experiments. Thanks for keeping the dream alive

quantumlump · 2024-11-07T04:27:31Z

Oh just saw your update! That's a massive increase! Dang that is so tantalizing especially because my card is so much faster than the 1050

CPU only? #1

CPU only? #1

Comments

quantumlump commented Nov 4, 2024 • edited Loading

DrewThomasson commented Nov 4, 2024 • edited Loading

About the piper-tts GPU use

About fine-tuning piper tts

Docs on it

Youtube Video on it

You say Hallucinations are the issue? I might have a better repo that doesn't hallucinate lol (Only limitation is that styletts2 only works for English) ebook2audiobookStyletts2

DrewThomasson commented Nov 4, 2024

ALSO You say you have some high quality xtts fine-tunes? 👀

Slap em on hugginface, send me links to them I'll check them out ! 😄

DrewThomasson commented Nov 4, 2024

I can confirm the fine-tuning piper-tts colabs still work as of this moment lol

Be aware it's very labor intensive for fine-tuning piper-tts 😭

quantumlump commented Nov 4, 2024 • edited Loading

quantumlump commented Nov 4, 2024 • edited Loading

DrewThomasson commented Nov 4, 2024 • edited Loading

Noted lol

I'm uploading the newest docker image for piper tts that supports GPU! :)

school wifi is slow 😅

DrewThomasson commented Nov 4, 2024 • edited Loading

quantumlump commented Nov 4, 2024

Noted lol

I'm uploading the newest docker image for piper tts that supports GPU! :)

school wifi is slow 😅

DrewThomasson commented Nov 4, 2024 • edited Loading

Yeah the notebooks are for opening in google colab

If you click on any of them you should see a button for "open in google colab" lol

DrewThomasson commented Nov 4, 2024

Here cause school wifi is too slow

DOCKER PIPER-TTS GPU

You can also duplicate the hugginface space if you want to give it a gpu from huggingface at like 40 cents an hour lol

quantumlump commented Nov 4, 2024

Here cause school wifi is too slow

DOCKER PIPER-TTS GPU

You can also duplicate the hugginface space if you want to give it a gpu from huggingface at like 40 cents an hour lol

quantumlump commented Nov 4, 2024

DrewThomasson commented Nov 4, 2024 • edited Loading

DrewThomasson commented Nov 4, 2024

Here, I tried fixing it by adding

Try again with this comand to make it force pull the latest docker image and see if this works ✨

DrewThomasson commented Nov 4, 2024 • edited Loading

I also created a free google colab of it with GPU support! :)

quantumlump commented Nov 4, 2024

DrewThomasson commented Nov 4, 2024 • edited Loading

OP you right, here try this for the google colab now I gave the wrong link

Or this:

quantumlump commented Nov 4, 2024

DrewThomasson commented Nov 4, 2024 • edited Loading

Yes I can confirm that the Google colab runs on GPU and gives you FREE GPUs

You don't need Colab Pro to use the GPUS of google colab

You can see here on the free version

quantumlump commented Nov 4, 2024

quantumlump commented Nov 4, 2024

DrewThomasson commented Nov 4, 2024

DrewThomasson commented Nov 4, 2024

Welp finally got it to work,

DrewThomasson commented Nov 4, 2024

DrewThomasson commented Nov 5, 2024

DrewThomasson commented Nov 5, 2024 • edited Loading

quantumlump commented Nov 5, 2024

DrewThomasson commented Nov 5, 2024

DrewThomasson commented Nov 5, 2024

DrewThomasson commented Nov 5, 2024

quantumlump commented Nov 5, 2024

DrewThomasson commented Nov 6, 2024

quantumlump commented Nov 6, 2024

DrewThomasson commented Nov 7, 2024 • edited Loading

DrewThomasson commented Nov 7, 2024

quantumlump commented Nov 7, 2024

quantumlump commented Nov 7, 2024 • edited Loading

quantumlump commented Nov 4, 2024 •

edited

Loading

DrewThomasson commented Nov 4, 2024 •

edited

Loading

You say Hallucinations are the issue?
I might have a better repo that doesn't hallucinate lol
(Only limitation is that styletts2 only works for English)
ebook2audiobookStyletts2

quantumlump commented Nov 4, 2024 •

edited

Loading

quantumlump commented Nov 4, 2024 •

edited

Loading

DrewThomasson commented Nov 4, 2024 •

edited

Loading

DrewThomasson commented Nov 4, 2024 •

edited

Loading

DrewThomasson commented Nov 4, 2024 •

edited

Loading

Yeah the notebooks are `for opening in google colab`

DrewThomasson commented Nov 4, 2024 •

edited

Loading

DrewThomasson commented Nov 4, 2024 •

edited

Loading

DrewThomasson commented Nov 4, 2024 •

edited

Loading

DrewThomasson commented Nov 4, 2024 •

edited

Loading

Yes I can confirm that the Google colab runs on `GPU` and gives you `FREE GPUs`

DrewThomasson commented Nov 5, 2024 •

edited

Loading

DrewThomasson commented Nov 7, 2024 •

edited

Loading

quantumlump commented Nov 7, 2024 •

edited

Loading