Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quantize before import and conversion to Linalg during CompiledModule.__new__() #211

Closed
wants to merge 1 commit into from

Conversation

renxida
Copy link
Contributor

@renxida renxida commented Nov 30, 2023

Now, passes that need to happen on torch IR before conversion to linalg can be supplied as pre_import_passes to the constructor of StateUpdateModule or other classes that inherit from CompiledModule.

E.g.

    pre_import_passes = []
    if quantization == "int4" and not compile_to == "linalg":
        from shark_turbine.transforms.quantization import mm_group_quant
        pre_import_passes.append(mm_group_quant.MMGroupQuantRewriterPass)
    inst = StateUpdateModule(context=Context(), import_to=import_to, pre_import_passes=pre_import_passes)

@renxida renxida changed the title makes quantization happen before import and conversion to linalg to fix #183 Quantize before import and conversion to Linalg during CompiledModule.__new__() Nov 30, 2023
@renxida
Copy link
Contributor Author

renxida commented Nov 30, 2023

This fixes #183

@IanNod
Copy link
Contributor

IanNod commented Nov 30, 2023

Please run black on the files changed

@IanNod IanNod linked an issue Nov 30, 2023 that may be closed by this pull request
@renxida renxida closed this May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate quantization pass with CompiledModule
2 participants