Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OV nf4 support #3209

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

OV nf4 support #3209

wants to merge 2 commits into from

Conversation

KodiaqQ
Copy link
Collaborator

@KodiaqQ KodiaqQ commented Jan 23, 2025

Changes

  • Added nf4 precision for OV GraphConverter.

Reason for changes

  • nf4 precision support.

Related tickets

  • 153357

Tests

  • N/A

@KodiaqQ KodiaqQ requested a review from a team as a code owner January 23, 2025 15:49
@github-actions github-actions bot added the NNCF OpenVINO Pull requests that updates NNCF OpenVINO label Jan 23, 2025
@KodiaqQ
Copy link
Collaborator Author

KodiaqQ commented Jan 23, 2025

@openvinotoolkit/nncf-maintainers, can someone review this? Thanks.

@ljaljushkin
Copy link
Contributor

Could you elaborate which scenario is supported now? Can it be covered by some test?

@KodiaqQ
Copy link
Collaborator Author

KodiaqQ commented Jan 23, 2025

Could you elaborate which scenario is supported now?

A model with partially quantized weights in nf4 format, as an example.

Can it be covered by some test?

I'm not sure that non-quantized models with these types exist. Only possible with the custom synthetic case. Do we really need it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ljaljushkin ljaljushkin Awaiting requested review from ljaljushkin

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
NNCF OpenVINO Pull requests that updates NNCF OpenVINO
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@KodiaqQ @ljaljushkin