CUDA OOM

[aishwary-intellifAI]

Hi,

Fantasy Portrait Demo Results on the repo seems amazing, I have tried reproducing those same and I’m getting a CUDA OOM while loading FantasyPortrait on an RTX 5090 (32GB) with Linux, CUDA 12.8, and PyTorch 2.7.

Tried lower number of frames at 420×420 resolution with num_persistent_param_in_dit=0 and 7*10**9 as mentioned here: https://huggingface.co/acvlab/FantasyPortrait#%F0%9F%93%A6speed-and-vram-usage, but still runs out of memory.

Any suggestions to fix this or run in a more memory-efficient mode?

Thanks.

[wangqiang9]

Hi, @aishwary-intellifAI
You can try using float8 in the code:

fantasy-portrait/infer.py

Line 261 in 3dd2665

# torch_dtype=torch.float8_e4m3fn, # You can set `torch_dtype=torch.bfloat16` to disable FP8 quantization.

This will be helpful for you to solve the GPU OOM problem.

[Pudge-tao]

4090 24GB vram by num_persistent_param_in_dit=5*10**9, max_size=512, num_frame=50. still runs out of memory.

[ruyijidan]

720P the vram use 50G-60G when num_persistent_param_in_dit=0,3s video 9min-12min