Discriminator variability on the same image when in different batches

[erezposner]

Hi, I've been using the discriminator of the trained NN as a black box to distinguish between generated and real image. My main goal is to define the range of the discriminator output values (as it is acts as a critic and not a classifier I understand it is not bound to a specific range but it would be nice to have an estimation of it).

My problem is that If I were to take an image and run it with different images in the batch it's output would change (I assume it has something to do with the minibatch normalization).

Also, If I set the batchsize to one, it seems as there is a large bias to the discriminator output.
for example:

• Running a single image usually result within the range -260 to -250. As you increase the batch size the results slowly getting bigger and bigger.

Is there a way to disable the affect of other images on each image result at inference?

[Molugan]

Hello,

Sorry for the delay. A good way to do this would be to get the average value of both the mean and the standard deviation batch of size subGroupSize (see here

pytorch_GAN_zoo/models/networks/mini_batch_stddev_module.py

Line 29 in b75dee4

y = torch.mean(y, 1).view(G, 1)

) on a set of images representative of the domain you want to work with (or at training time if you train the model by yourself) and use it to get your scores.

[erezposner]

What do you mean? I actually did something pretty similar to what you suggested. That is to hand pick a control group (or a set of images representative of the domain) and then just change my one sample to be tested.

In my approach I always use the same mini batch with just changing the sample. How can I use the mean and std on a single sample then?

[Molugan]

You need to compute the mean and std on a selected "training" set and uses theses values for each each test iteration. See for example the code of batchNorm on pytorch https://pytorch.org/docs/master/_modules/torch/nn/modules/batchnorm.html#BatchNorm2d

[gengcong940126]

I set the self.gan.netD.eval(),but it also has this problem.