Question Regarding the Combination of DMT and TSIT

[DDKdakuan]

We observed that the authors effectively combined DMT with TSIT in both the paper and the code, achieving impressive overall results for DMT. During the "inference" stage, we noticed that for the TSIT model, both noisy_input_semantics and noisy_real_image are used to estimate the fake_image. However, my question is that during the "inference" stage, it seems that noisy_real_image cannot be directly obtained. Could the authors kindly elaborate on how this was considered and handled in this specific context?

In dmt_model line46
“elif mode == 'inference':
with torch.no_grad():
fake_image, _ = self.generate_fake(noisy_input_semantics,
noisy_real_image)
batch_size = fake_image.shape[0]
”

[thuxmf]

Thanks for following our work! We directly modify the codes from TSIT without any structural changes. In TSIT, the generator will be fed in with a sample from target domain for style extractor. Therefore, it would be proper to apply similar input.

[DDKdakuan]

Thank you so much for your detailed response! I have an additional question—when using TSIT and DMT, do TSIT and pix2pix rely solely on a generator, or do they also incorporate a discriminator, similar to the traditional GAN training process?