Log-reward uniformisation, logging and new hyperparams for temperature, Z-training, validation. #54

julienroyd · 2023-03-10T15:20:26Z

Several changes (most of them minor):

Added option ‘constant’ for ‘temperature_sample_dist’
New hyperparameters:
- objectives (variable list of objectives to optimise)
- num_thermometer_dim (for beta encoding)
- Z_learning_rate
- valid_random_action_prob (now performs greedy_validation by default)
- n_valid_prefs (number of validation preference vector to span simplex)
- n_valid_repeats_per_pref (number of molecules to sample for each preference vector at validation)
Added logging
- Now saves molecules generated during validation
- Saves hyperparameters in stdout and on disk
- Saves ‘git_hash’ and ‘hostname’ in hyperparameters
- To stdout during training
Made all algorithms consistent with log-rewards

…de original fragment stems

…spond to self.temperature_dist_params=<fixed_value>

…ode_conditional_information()

* objectives: the list of objectives to optimise among {'seh', 'qed', 'sa', 'mw'} * num_thermometer_dim: the length of the thermometer vector for encoding temperature

…vmax=0 when beta=0

…on Mac M1 chip

…passing full thermometer instead of empty one)

…on_prob to be a argument of graph_sampler.sample_from_model()

* n_valid_prefs * n_valid_repeats_per_pref

….setup() to be caught by Determined agents as well

…PR39)

…ling task.cond_info_to_reward() for tasks that wouldn't have been updated to using logrewards rather than rewards)

…sks: * SEHMOOTask now inherits from SEHTask * Now SEHMOOTask only concatenates the preferences to what was already in cond_info (done by parent classes)

…tuple(float, float) or a float, parsing a string (say if provided by Determined) should be handled by an external wrapper.

src/gflownet/algo/soft_q_learning.py

src/gflownet/tasks/qm9/qm9.py

src/gflownet/utils/metrics.py

src/gflownet/tasks/seh_frag.py

…tox checks)

…all epsilon)

bengioe · 2023-03-20T18:17:29Z

#50 and #52 were merged, would you mind resolving conflicts and rebasing?

julienroyd · 2023-03-20T20:22:22Z

@bengioe I merged the trunk here, we can fast-forward now. I am currently running an experiment just to validate that we still get the same results and then we should be good to close. I'll let you know when the results are in.

julienroyd · 2023-03-22T14:51:26Z

I am done with the validation run, results look good (see on this link).

julienroyd · 2023-03-22T14:51:42Z

Closing.

bengioe and others added 30 commits February 18, 2023 20:07

improve fragment environment for validity + include stop_mask + inclu…

c6b5018

…de original fragment stems

tox

7df66de

remove temp code

b5ebcb7

Remove json prettifier pre-commit hook

014cb41

fix mask to device in GraphActionCategorical

e5e7939

more comments and slightly better architecture adaptation

cb1b099

fix: removed np.int (deprecated)

f4fdfa7

feat: added option hps.temperature_sample_dist='constant' which corre…

4fe0b36

…spond to self.temperature_dist_params=<fixed_value>

fix: added case for self.temperature_sample_dist == 'constant' in enc…

2a68454

…ode_conditional_information()

feat: added hyperparams 'objectives' and 'num_thermometer_dim'

10196b9

* objectives: the list of objectives to optimise among {'seh', 'qed', 'sa', 'mw'} * num_thermometer_dim: the length of the thermometer vector for encoding temperature

fix: use 'objectives' hyperparam for validation set construction

05dafc0

fix: added security-check to thermometer and avoid passing both vmin=…

1d74a1d

…vmax=0 when beta=0

feat: added a logger to run()

b64a26b

fix: isolating sensitive imports (cvxopt) that are tricky to install …

5759fc9

…on Mac M1 chip

(fix): fixed encode_conditional_information() for constant beta (was …

a4af792

…passing full thermometer instead of empty one)

feat: added hyperparameter 'Z_params'

02ece4a

fix: removed batch_size*2 for training sampling_iterator

4a7aae0

(minor): added prints

7279d97

feat: added argument 'valid_random_action_prob' and moved random_acti…

2fefa06

…on_prob to be a argument of graph_sampler.sample_from_model()

Added hyperparameters:

a2be0a0

* n_valid_prefs * n_valid_repeats_per_pref

docs: added hyperparmeters in printing and in logs as json file

3273b2c

chore: moved config saving from GFNTrainer.run() to SEHMOOFragTrainer…

2978fd4

….setup() to be caught by Determined agents as well

feat: added git_hash to saved hyperparams

67bc51d

feat: added 'hostname' in hyperparameters for cluster troubleshooting

a61e584

feat: use log-rewards for numerical stability (forgotten commit from …

57f24a1

…PR39)

docs: renamed functions to logrewards (to avoid TrajectoryBalance cal…

4e94328

…ling task.cond_info_to_reward() for tasks that wouldn't have been updated to using logrewards rather than rewards)

fix: convert logrewards back into regular rewards for logging

bc9e12f

Merge branch 'fix_frag_env' into julien-general-pr

7503f61

chore: pep8

1a5fc6d

Added support for self.temperature_sample_dist='constant' to other ta…

a3e5ac0

…sks: * SEHMOOTask now inherits from SEHTask * Now SEHMOOTask only concatenates the preferences to what was already in cond_info (done by parent classes)

julienroyd added 6 commits March 9, 2023 09:33

chore: updated hyperparameters for seh_frag_moo.py and seh_frag.py

281a049

fix: more shape-checks and squeeze 1-D 'flat_rewards' in seh_frag.py

b207360

feat: added hyperparam 'overwrite_existing_exp' for debugging purposes

b3d589d

fix: added self.rng attribute to tasks

e2bc169

refactor: removing ast.literal(temperature_dist_params). Should be a …

988809c

…tuple(float, float) or a float, parsing a string (say if provided by Determined) should be handled by an external wrapper.

chore: pep8

ec8d245

julienroyd requested a review from bengioe as a code owner March 10, 2023 15:20

bengioe reviewed Mar 13, 2023

View reviewed changes

src/gflownet/algo/soft_q_learning.py Outdated Show resolved Hide resolved

bengioe reviewed Mar 13, 2023

View reviewed changes

src/gflownet/tasks/qm9/qm9.py Outdated Show resolved Hide resolved

bengioe reviewed Mar 13, 2023

View reviewed changes

src/gflownet/utils/metrics.py Show resolved Hide resolved

bengioe reviewed Mar 13, 2023

View reviewed changes

src/gflownet/tasks/seh_frag.py Outdated Show resolved Hide resolved

julienroyd added 5 commits March 14, 2023 12:19

fix: corrected hps fetch in SQL

7e6893d

fix: exposing bounds of uniform distribution for mypy

9348a66

fix: re-create log_dir path if overwriting past results

1a95253

fix: added timeout=30 in load_original_model() to satisfy bandit (in …

cb4bba4

…tox checks)

chore: adding comment about non-top-level import for cvxopt

b3c1238

julienroyd force-pushed the julien-pr1 branch 2 times, most recently from 9b8f7ee to 48198cf Compare March 15, 2023 15:50

fix: clamp rewards before taking into log-space (rather than add a sm…

e3fdb25

…all epsilon)

julienroyd force-pushed the julien-pr1 branch from 48198cf to e3fdb25 Compare March 15, 2023 15:52

Merge branch 'trunk' into julien-pr1

d96988d

julienroyd added 2 commits March 20, 2023 14:53

chore: removed frags_72.json

1580ae8

fix: re-include commented-out hook in pre-commit

297d84f

bengioe approved these changes Mar 21, 2023

View reviewed changes

julienroyd merged commit 838bbd6 into trunk Mar 22, 2023

julienroyd deleted the julien-pr1 branch March 22, 2023 14:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Log-reward uniformisation, logging and new hyperparams for temperature, Z-training, validation. #54

Log-reward uniformisation, logging and new hyperparams for temperature, Z-training, validation. #54

julienroyd commented Mar 10, 2023

bengioe commented Mar 20, 2023 •

edited

Loading

julienroyd commented Mar 20, 2023

julienroyd commented Mar 22, 2023

julienroyd commented Mar 22, 2023

Log-reward uniformisation, logging and new hyperparams for temperature, Z-training, validation. #54

Log-reward uniformisation, logging and new hyperparams for temperature, Z-training, validation. #54

Conversation

julienroyd commented Mar 10, 2023

bengioe commented Mar 20, 2023 • edited Loading

julienroyd commented Mar 20, 2023

julienroyd commented Mar 22, 2023

julienroyd commented Mar 22, 2023

bengioe commented Mar 20, 2023 •

edited

Loading