Concern about running test.sh

[CherryNhu]

Hello,

Thank you for providing the resources and instructions in the README. I have carefully prepared the environment following the link you shared, and I used the checkpoints you provided to run test.sh without modifying any related code.

However, I noticed that the results I obtained on the three datasets are significantly lower than the ones reported in your paper. To better understand and reproduce your results, would it be possible for you to share the .arrow files of the three datasets used for testing?

I appreciate your help and look forward to your response.

[GaryGuTC]

Hi,

Thank you for reaching out.

Regarding the test arrow file you mentioned:

• The file was originally downloaded from the M3AE repository
• We followed the same preparation steps as outlined in M3AE's documentation, so we should be working with identical data
• Unfortunately, due to workplace changes, I'm currently unable to provide the specific file - I sincerely apologize for this inconvenience

About the result differences:

• From my experience, the seed number typically has minimal impact on final results
• There might be other factors contributing to the variance that we could explore

Best regards