Question on Fig 3

[Haichao-Zhang]

Thanks for the great work.

I'm trying to understand the curves in Fig 3, (CALVIN-ABC-D), especially on the "scratch" version (also the "abc_train_partial_xx_percent" in https://drive.google.com/drive/folders/1RqwIv5XGaFkM5EUyg85A2noL3Rk5ihz8?usp=drive_link)

For the "scratch" version in Fig 3 (i.e. "Ours (w/o Pre-training)"):

1. is it correct that we only used the part of CALVIN-ABC that has language annotations for training?
2. And the data proportion of the x-axis is on top of that (e.g. 20% means 20% out of the part of CALVIN-ABC data with language annotation)?

Thanks!

[Nimolty]

Dear Haichao-Zhang,

We utilize the CALVIN ABC dataset with language annotations for post-training.

"20%" refers to 20% of the language-annotated portion of the CALVIN ABC dataset.

All of your assumptions are correct.

Best Regards,
Yang Tian
2026.1.9

[Haichao-Zhang]

@Nimolty I see. Thanks for the explanation.

Is possible for you to share the eval logs of the "scratch" version (lang only) at the 100% Data Portion?
As I can see the logs for 10%,~70%, but not the final 100% one (e.g. abc_train_partial_100_percent) from the shared folder:
https://drive.google.com/drive/folders/1RqwIv5XGaFkM5EUyg85A2noL3Rk5ihz8

[Haichao-Zhang]

@Nimolty I see. Thanks for the explanation.

Is possible for you to share the eval logs of the "scratch" version (lang only) at the 100% Data Portion? As I can see the logs for 10%,~70%, but not the final 100% one (e.g. abc_train_partial_100_percent) from the shared folder: https://drive.google.com/drive/folders/1RqwIv5XGaFkM5EUyg85A2noL3Rk5ihz8

Hi Yang (@Nimolty ),

Simply want to follow up on the above question (eval logs of scratch version at 100% Data Portion). If you still have it, is it possible for you to share it as well? Thanks!