RecursionError

[fm-frga]

I got a recursion error from running the fine-tuning.

Traceback (most recent call last):
  File "E:\VSCODE\MyCondaEnvs\DLAS\lib\site-packages\torch\utils\data\_utils\worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "E:\VSCODE\MyCondaEnvs\DLAS\lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "E:\VSCODE\MyCondaEnvs\DLAS\lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "E:\VSCODE\MyCondaEnvs\DL-Art-School\codes\data\audio\paired_voice_audio_dataset.py", line 222, in __getitem__
    return self[rv]
  File "E:\VSCODE\MyCondaEnvs\DL-Art-School\codes\data\audio\paired_voice_audio_dataset.py", line 222, in __getitem__
    return self[rv]
  File "E:\VSCODE\MyCondaEnvs\DL-Art-School\codes\data\audio\paired_voice_audio_dataset.py", line 222, in __getitem__
    return self[rv]
  [Previous line repeated 977 more times]
  File "E:\VSCODE\MyCondaEnvs\DL-Art-School\codes\data\audio\paired_voice_audio_dataset.py", line 207, in __getitem__
    return self[(index+1) % len(self)]
  File "E:\VSCODE\MyCondaEnvs\DL-Art-School\codes\data\audio\paired_voice_audio_dataset.py", line 207, in __getitem__
    return self[(index+1) % len(self)]
  File "E:\VSCODE\MyCondaEnvs\DL-Art-School\codes\data\audio\paired_voice_audio_dataset.py", line 207, in __getitem__
    return self[(index+1) % len(self)]
  [Previous line repeated 7 more times]
  File "E:\VSCODE\MyCondaEnvs\DL-Art-School\codes\data\audio\paired_voice_audio_dataset.py", line 250, in __len__
    return len(self.audiopaths_and_text)
RecursionError: maximum recursion depth exceeded while calling a Python object

The GPT4 response is.
The error you're encountering is a RecursionError: maximum recursion depth exceeded. It seems to be occurring in the getitem method of your paired_voice_audio_dataset.py file. This error usually occurs when a function calls itself too many times, leading to a stack overflow.

Here's what's happening in your code:

When the getitem method encounters an exception (in this case, the RecursionError from the tokenizer), it attempts to recover by calling itself with a new index (return self[rv]).
This recovery process is repeated recursively until the maximum recursion depth is reached.
To fix this issue, you should refactor the getitem method to avoid recursion. One possible solution is to use a loop to retry a fixed number of times before raising an error. Here's an example of how you could modify the getitem method:

then it suggest this fix
`def getitem(self, index):
retry_count = 0
max_retries = 10 # You can set a different value depending on your needs.

while retry_count < max_retries:
    try:
        tseq, wav, text, path, type = self.get_wav_text_pair(self.audiopaths_and_text[index])
        return tseq, wav, text, path, type
    except RecursionError:
        rv = random.randint(0, len(self) - 1)
        index = rv
        retry_count += 1

raise RuntimeError(f"Failed to load data after {max_retries} retries.")

I don't really trust GPT on generating a fix for it. Has anyone encounter a similar error?

[152334H]

this basically happens when too many of the input audio files are rejected for being invalid. the dataset sampler is a bit jank; it simply recursively calls itself for a new item when the current item is invalid, and that causes a recursionerror if it sees dozens of bad files at once.