Visualizing Latent points before decoding?

[Javiergonb]

Hi again @ZENGXH,

Is there a way to visualize the z_local tensor that is created when running the DDPM on the encoded h0?
When using the sample method of the LION model the output dict returns a z_local with (num_samples, 8192,1,1) dimensions. From my understanding this is the z_local after decoding? I am not sure how visualize in 3D the z_local that is created before the decoding, like how the show here:

How did you do it?

Thanks in advance!

[ZENGXH]

could you try to reshape it to z_reshape in (num_samples, 2048, 4,1,1) and then visualize z_reshape[:,:,:3] (the last channel is feature not xyz, so we dont plot it)? Let me know if it does not work

[Javiergonb]

Yes this works. I was trying to do something similar with the VAE. I am getting h0 from the recont method in the VAE model. This part of the code:

https://github.com/nv-tlabs/LION/blob/333c5b4903098418a2aae897ff079bcf0adc3eae/models/vae_adain.py#L202C16-L202C99

I believe this is getting a 3d representation of h0 no?