EF-Net concatenate zt question

[jasonlovechu]

Thanks for your job！ I want to ask that in your paper, you depict that you concatenate the zt which is the noised latent after linear projection in EF-Net, but in the inference code, what you actually do is concatenate the latent_image_input, which is first + 0 + end. What are the actual settings in training, to concatenate the noisy ground truth image or only the start and end frame with 0 padding?

[LiuhanChen-github]

Thank you for your attention. The actual training setting is what I depict in the paper, concatenate zt (the noisy ground truth video).

[LiuhanChen-github]

The part of the training code related to this question is here, and we will release the training code soon.

[jasonlovechu]

ok, I got it. So EF-Net actually take noisy video latent during training and start+0+end during inference stage. Thx for your replication.

[LiuhanChen-github]

This is a version, actually, for inference, the noisy video latent input is also ok. The key of our method is to enhance the injection of the end frame constraint.

[LiuhanChen-github]

I have updated the inference code that I changed the start+0+end to zt.