Problem: Multiple higher Cuda Reserved memory than allocated memory for PatchCore

[Cottoneyejoe95]

Hello everyone,

first of all thank you for the great repo, it helps me a lot to do my research. :)
I experimented with PatchCore regarding different input sizes and their consequences regarding VRAM usage.

During these experiments I saw that sometimes for the coreset subsampling step and during inference distance calculation the reserved memory is multiple times higher than the allocated memory. This is of course a very challenging behavior when you are limited regarding the available VRAM on your GPU and you want to make assumptions on required VRAM for use-cases with varying image- and dataset-sizes.

Some extreme examples:
Image size = 3250x3250 pixels; dataset-size = 1 --> Leads to 1641 MB allocated, but 5426 MB of reserved VRAM during coreset subsampling
Image size = 1700x1700 pixel; dataset-size = 1 --> Leads to 511 MB allocated, but 1568 MB of reserved VRAM in during coreset subsampling

In other examples where the Image size is smaller:
Image size = 400x400 pixels; dataset-size = 100 --> Leads to 2451MB allocated, but 4092 MB of reserved VRAM during coreset subsampling.

In my experiments the ratio betwenn allocated VRAM and reserved VRAM differs between 1,19 and up to 3,3 during coreset subsampling as well as during inference time distance calucation.

From my experiments the rule of thumb would be:
The higher the image- to dataset-size ratio the higher the reserved memory factor or coreset subsampling.
The higher the dataset- to image-size ratio the higher the reserved memory factor for distance calucation

Did anybody else see that behavior yet? Does anybody of you guys have any idea why this happens during coreset subsampling and distance calculcation?