Does TorchIO have its way of controlling randomness/reproducibility?

[dzenanz]

In order to have the same results each time my code is run, is general pytorch discussion the most valid, or does TorchIO has its own (easier) way of controlling it?

[romainVala]

Hi ,
I think all random state of torchio Random transform comes from numpy , so using
np.random.seed(0) at the begining of your code, should be ok.

but the dataselection (and randomness) comes from torch dataloader, so you should also add
torch.manual_seed(0)

[dzenanz]

Is worker_init_fn=seed_worker needed when num_workers is greater than 0?

[fepegar]

@romainVala

Hi ,
I think all random state of torchio Random transform comes from numpy , so using
np.random.seed(0) at the begining of your code, should be ok.
but the dataselection (and randomness) comes from torch dataloader, so you should also add
torch.manual_seed(0)

NumPy is not used for random operations, so torch.manual_seed is what should be used.

---

@dzenanz

Is worker_init_fn=seed_worker needed when num_workers is greater than 0?

I don't think so, but I'm not sure.

Is worker_init_fn=seed_worker needed when num_workers is greater than 0?

No, I don't think so.

[dzenanz]

Yesterday, I added

np.random.seed(30101983)
torch.manual_seed(30101983)

I ran my training overnight, now I am re-running it to see whether I get the same results again.

Numbers are similar now, but not exactly the same. I will try to also add random.seed(30101983).

[fepegar]

Thanks for reporting. Did you also used torch.use_deterministic_algorithms?

[dzenanz]

No. I will try that.

Adding random.seed() improves reproducibility, the numbers are noticeably closer but still not exactly the same.

[dzenanz]

That did it. Adding the following code to my program made it fully reproducible across runs:

random.seed(30101983)
np.random.seed(30101983)
torch.manual_seed(30101983)
torch.use_deterministic_algorithms(True)

[dzenanz]

Thanks both @fepegar and @romainVala!

[dzenanz]

TL;DR version: Adding this made the program fully reproducible across runs:

random.seed(30101983)
np.random.seed(30101983)
torch.manual_seed(30101983)
torch.use_deterministic_algorithms(True)

[fepegar]

Thanks, @dzenanz! I have opened a PR to remove the shuffle using random, so, in theory, only the last two lines should be relevant for TorchIO:

• Use PyTorch to shuffle the queue patches #776