comparison between gumbel softmax, rebar, and relax

[fnzhan]

Hi, @HEmile , I test the example in examples/vae/discrete_vae.py, but find the gumbel softmax performs much better than rebar, relax and reinforce (testing loss after 10 epochs: 98 for gumbel, 165 for rebar, 208 for relax, 209 for reinforce). The performance is not that consistent with the results in your guide, where gumbel softmax is slightly worse than reinforce in terms of training loss. Is there anything wrong with the example? I don't change any parameters. Looking forward to your response.

[fnzhan]

Besides, I also test the sample in examples/vae_reference.py, still find Gumbel outperform reinforce and rebar. (n_samples=10, final loss: 122 for Gumbel, 157 for reinforce, 152 for rebar).

[HEmile]

Hi @fnzhan . That is a good question! I wrote that tutorial a while ago, and am surprised by the results myself. I think the point here is that I used a latent space of 2 discrete latents with 10 categories. Apparently, back then, Gumbel softmax did rather poorly.
In general I'd expect Gumbel to outperform REINFORCE, REBAR and RELAX if it is applicable.
Did you test with a latent space of 10x10 discrete latents (default in the example code) or 2x10?

[fnzhan]

Hi @HEmile Thanks for your reply. In my test, the latent space size in examples/vae/discrete_vae.py and examples/vae_reference.py are 20×10 and 2×10 (i.e., the default setting), respectively.
I run the examples/vae_reference.py with n_sample=10, while the n_sample is 1 in your tutorial. Thus, I guess the inconsistency for examples/vae_reference.py is caused by the different n_sample.
For examples/vae/discrete_vae.py, I run with n_sample=1.

In all, it seems REINFORCE outperforms Gumbel only when there is a small latent space (e.g., 20) and small n_sample (e.g., 1). Otherwise, Gumbel may outperform REINFORCE and REBAR consistently. Above is only for discrete VAE.

[HEmile]

Yes, I think that is consistent with my results. I will update the documentation to make this clearer.