pickling errors in torch.load when checkpoints are downloaded via curl #103
Description
Wondering if you get pickling errors as below when using downloaded checkpoints via option 2:
usr/local/lib/python3.10/dist-packages/torch/utils/_pytree.py:185: FutureWarning: optree is installed but the version is too old to support PyTorch Dynamo in C++ pytree. C++ pytree support is disabled. Please consider upgrading optree using `python3 -m pip install --upgrade 'optree>=0.13.0'`.
warnings.warn(
Using device: cuda
/mnt/graid/biomedparse/BiomedParse/run_new_example_from_config.py:16: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
hydra.initialize(config_path="configs/model", job_name="example_prediction")
/usr/local/lib/python3.10/dist-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'biomedparse_3D': Defaults list is missing `_self_`. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information
warnings.warn(msg, UserWarning)
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:942: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
warnings.warn(
Traceback (most recent call last):
File "/mnt/graid/biomedparse/BiomedParse/run_new_example_from_config.py", line 19, in <module>
model.load_pretrained("model_weights/biomedparse_v2.ckpt")
File "/mnt/graid/biomedparse/BiomedParse/src/model/biomedparse_3D.py", line 393, in load_pretrained
checkpoint = torch.load(checkpoint_path, map_location="cpu", weights_only=True)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1494, in load
raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
_pickle.UnpicklingError: Weights only load failed. In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
Please file an issue with the following so that we can make `weights_only=True` compatible with your use case: WeightsUnpickler error: Unsupported operand 65
I have simply used the curl command and then changed this line to point to my on disk ckpt
model.load_pretrained("model_weights/biomedparse_v2.ckpt")
Also, changed the line to include weights_only, however that did not help.
checkpoint = torch.load(checkpoint_path, map_location="cpu", weights_only=True)