[manyblock mode] why store embedding in "xt_sh" if we are going to overwrite it anyways?

[isaacleeai]

in function "nv_wavenet_persistent_cur", we add values from "embedPrev" and "embedCur". Then we save the values in "xt_sh" and the same values in "xt". Then, before the GEMM, we update the values in "xt_sh" again ( if layer == 0, then we are storing values that are exactly the same as the values they are replacing ). So is it necessary to store the added embedding values in "xt_sh" in the first place?

[isaacleeai]

Tested the above hypothesis:
in nv_wavenet_persistent_cur in nv_wavenet_persistent.cuh, modified the part where we load values from Xt and a_prev. Instead of checking for both Xt and a_prev, I put a conditional s.t. it does not check for Xt and only checks for a_prev when layer == 0. Passed nv_wavenet_test with manyblock mode only ( but since persistent is almost like manyblock, they should both work, although I can't test persistent on the GPU I have ).

Please let me know if I am wrong or share your thoughts.

Thanks