Fix the bug that MAISI ckpt cannot be loaded after finetune. #654
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes # .
Description
MAISI output checkpoint after finetuning cannot be used as
trained_controlnet_path
.This problem came from the
CheckpointSaver
. When a single key is provided forsave_dict
such as:"save_dict": {
"controlnet_state_dict": "@ControlNet"
},
The saved dict does contain the key " "controlnet_state_dict". However, it directly saves the state_dict of controlnet as the checkpoint.
The workaround is that we also save the optimizer state. For example,
"save_dict": {
"controlnet_state_dict": "@ControlNet",
"optimizer": "@optimizer"
}. Then, the MAISI output checkpoint after fine-tuning can be properly loaded.
Status
Ready/Work in progress/Hold
Please ensure all the checkboxes:
./runtests.sh --codeformat
.version
andchangelog
inmetadata.json
if changing an existing bundle.CONTRIBUTING.md
).monai
,pytorch
andnumpy
are correct inmetadata.json
.eval_metrics
of the provided weights and TorchScript modules.large_file.yml
./home/your_name/
for"bundle_root"
).