How much RAM do you recommend?

[ghost]

Hi,

I was wondering how much CUDA Memory do you recommend for running D4, D5, D6, D7 and D7X? I have 16GB of CUDA memory available, should this be enough?

I can run D0, D1, D2 and D3 fine but after that I don't have enough CUDA memory anymore. I already reduced my batch size to 1 and I have set torch.backends.cudnn.benchmark to False. I have step through the code but something within the network causes the RAM to go way up.

[ghnreigns]

@dylanhiemstra Hi I’m also interested to know! Btw for your d3 model, are you able to provide me on whether a 10gb vram 3080 is sufficient to run a batch size of 4? I’m not sure whether to invest on a 3090

[ghost]

@dylanhiemstra Hi I’m also interested to know! Btw for your d3 model, are you able to provide me on whether a 10gb vram 3080 is sufficient to run a batch size of 4? I’m not sure whether to invest on a 3090

Hi. So I am using tiling and for D3 I have a batch size of 1 (meaning 2 tiles in the mini batch) and it is using 14487MiB RAM. But I am not sure if I can disable or enable anything to reduce the RAM. So to answer your question: probably not? We will have to see :).

[ghnreigns]

@dylanhiemstra Hi I’m also interested to know! Btw for your d3 model, are you able to provide me on whether a 10gb vram 3080 is sufficient to run a batch size of 4? I’m not sure whether to invest on a 3090

Hi. So I am using tiling and for D3 I have a batch size of 1 (meaning 2 tiles in the mini batch) and it is using 14487MiB RAM. But I am not sure if I can disable or enable anything to reduce the RAM. So to answer your question: probably not? We will have to see :).

Thanks for the reply! Interesting! When I was using Efficientdet d6 with a batch size of 4 on Kaggle’s GPU which has a 16gb vram, it seems to run smoothly. But I notice on a local GPU it is not the case even if you have 16gb. May I know what gpu model you are using?

[ghost]

@dylanhiemstra Hi I’m also interested to know! Btw for your d3 model, are you able to provide me on whether a 10gb vram 3080 is sufficient to run a batch size of 4? I’m not sure whether to invest on a 3090

Hi. So I am using tiling and for D3 I have a batch size of 1 (meaning 2 tiles in the mini batch) and it is using 14487MiB RAM. But I am not sure if I can disable or enable anything to reduce the RAM. So to answer your question: probably not? We will have to see :).

Thanks for the reply! Interesting! When I was using Efficientdet d6 with a batch size of 4 on Kaggle’s GPU which has a 16gb vram, it seems to run smoothly. But I notice on a local GPU it is not the case even if you have 16gb. May I know what gpu model you are using?

Hmm. That's weird! I am using Tesla P100-SXM2. I am not familiar with Kaggle’s GPU, what is the difference?

[ghnreigns]

@dylanhiemstra Hi I’m also interested to know! Btw for your d3 model, are you able to provide me on whether a 10gb vram 3080 is sufficient to run a batch size of 4? I’m not sure whether to invest on a 3090

Hi. So I am using tiling and for D3 I have a batch size of 1 (meaning 2 tiles in the mini batch) and it is using 14487MiB RAM. But I am not sure if I can disable or enable anything to reduce the RAM. So to answer your question: probably not? We will have to see :).

Thanks for the reply! Interesting! When I was using Efficientdet d6 with a batch size of 4 on Kaggle’s GPU which has a 16gb vram, it seems to run smoothly. But I notice on a local GPU it is not the case even if you have 16gb. May I know what gpu model you are using?

Hmm. That's weird! I am using Tesla P100-SXM2. I am not familiar with Kaggle’s GPU, what is the difference?

I’m not too good with gpu and it’s memory issues. But as far as I know Kaggle uses Tesla P100 GPUs. Not too sure how different their one is compared to your P100 though.

[rwightman]

No, 16GB will not be enough to train those larger models at their native resolution, I can't do it in 24GB. If you're fine tuning maybe you'll have some luck with small batch sizes, but training from scratch reliably with less than 6-8 batch size is pretty hard.