Skip to content

Conversation

@christopherzc
Copy link
Collaborator

@christopherzc christopherzc commented Sep 13, 2025

The purpose of this pull request is to add the training and testing splits to TALES.

  • All frameworks outside of Jericho now expose a 'train_environments' variable that contains the environment name and version number for the training split of all synthetic frameworks. [1], [2], 3, 4
  • A README is added to the 'tales' folder demonstrating how to initialize these training environments. [5]
  • A 'step 0' has been added to main README directing users to README in tales folder for instructions on how to initialize gym envs. [6]

Specific changes:

  • For Alfworld and Textworld, we change the ALFWorldTask/TWCookingEnv init signature to take in all game files and a start game file. [7], [8]
    • This is so that in the init for these frameworks, we can pull all of the game files and split them into train game files and test game files, and register these with gym. [9], [10]
    • As we pull the game files in the init, we no longer need to do it in ALFWorldTask/TWCookingEnv.
      • For the testing environment specifically, we enforce only allowing the test env to use the same game files that were used for evaluating the LLMs in the technical report. [11], [12]
  • For Textworld Express and Scienceworld, we only add the ability to pass a 'split' to the environment and subsequently also register the training split with gym. [13], [14][15]
    • Should we also pin the version for the testing version here too?

christopherzc and others added 3 commits October 22, 2025 03:46
Signed-off-by: Christopher Zhang Cui <czcui@ucsd.edu>
Signed-off-by: Christopher Zhang Cui <czcui@ucsd.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants