How to improve out of memory problem?

[uyen-vuong]

Testing 3 4
Error CUDA out of memory. Tried to allocate 4.63 GiB (GPU 0; 23.64 GiB total capacity; 10.75 GiB already allocated; 3.48 GiB free; 17.84 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF 4

[DENGBOYU-REX]

Same here. Did you find out the reason?

[zhaohm14]

Hi! I tried --tile 512 and it works fine on my 2080Ti, using 17G/22G of VRAM.

[ming053l]

Hi, based on our follow-up experimental experience regarding memory insufficiency issues, in addition to reducing the MLP dimension or number of heads, you may consider trying to use 4 STLs in RDG instead of the 5 provided in the paper. Due to the dense connection, stacking each STL requires more parameters (180+32×number of STLs). Therefore, reducing from 5 to 4 STLs will significantly decrease the parameter-size and memory requirements; meanwhile, this modification only has a minor impact on the model's prediction performance. [image: image.png] zhaohm14 ***@***.***> 於 2025年1月20日 週一 下午2:55寫道：

…

Hi! I tried --tile 512 and it works fine on my 2080Ti, using 17G/22G of VRAM. — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A4P3JL2XUBQUPYVOA5YQ3WD2LSMVVAVCNFSM6AAAAABMRQ3TCOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMBRGUYDKOBXGU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>