Skip to content

Conversation

tboerstad
Copy link
Contributor

PR #445 introduced a direct import of has_offloaded_params from accelerate.utils at the module level in transform/apply.py.

This could lead to upstream projects, like SGLang, failing to run with:

  File ".../compressed_tensors/transform/apply.py", line 18, in <module>
    from accelerate.utils import has_offloaded_params
ModuleNotFoundError: No module named 'accelerate'

This PR just changes to has_offloaded_params from compressed_tensors.utils.offload instead of accelerate.utils directly. This ensures accelerate remains optional and the package can be imported without it installed.

The utils.offload module already has proper fallback handling with @check_accelerate decorator that returns False when accelerate is not available.

I have run basic tests checking that imports work, and also the tests/test_transform/. I'm new to this project, so please let me know if there's something else needed.

Import has_offloaded_params from compressed_tensors.utils.offload instead of
accelerate.utils directly. This ensures accelerate remains optional and the
package can be imported without it installed.

The utils.offload module already has proper fallback handling with
@check_accelerate decorator that returns False when accelerate is not available.

Fixes ModuleNotFoundError when importing compressed_tensors without accelerate.
@tboerstad
Copy link
Contributor Author

To the maintainers:

The .yaml workflows (except for quality-check.yaml) all install [dev,accelerate] iiuc, so this was not caught in CI.
You could consider running some tests/or just import compressed_tensors in a CI job which only does not install any extras to avoid this happening in the future.

I don't want to mess with the CI as I'm new

Copy link
Contributor

@kylesayrs kylesayrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome catch, thank you!

@kylesayrs
Copy link
Contributor

kylesayrs commented Oct 3, 2025

@tboerstad You're absolutely right. This is somewhat difficult to catch as of now because the CI usually installs accelerate. At the moment it's probably not worth running additional CI runs without accelerate, so I'm glad this was caught.

@dsikka dsikka merged commit 6625cd3 into neuralmagic:main Oct 3, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants