Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PT2] MinMax #3166

Merged
merged 28 commits into from
Jan 29, 2025
Merged

[PT2] MinMax #3166

merged 28 commits into from
Jan 29, 2025

Conversation

AlexanderDokuchaev
Copy link
Collaborator

@AlexanderDokuchaev AlexanderDokuchaev commented Dec 23, 2024 •
edited
Loading

Changes

Introduce TORCH2 backend
MinMax algorithms for torch2 backend
Add handle_torch_function for quantization function to trace it by torch_function

Related tickets

152996

Tests

test install

@github-actions github-actions bot added NNCF PT Pull requests that updates NNCF PyTorch NNCF Common Pull request that updates NNCF Common experimental NNCF PTQ Pull requests that updates NNCF PTQ API Public API-impacting changes labels Dec 23, 2024
@AlexanderDokuchaev AlexanderDokuchaev marked this pull request as ready for review December 25, 2024 09:58
@AlexanderDokuchaev AlexanderDokuchaev requested a review from a team as a code owner December 25, 2024 09:58
daniil-lyakhov
daniil-lyakhov reviewed Dec 29, 2024
View reviewed changes
Copy link
Collaborator

@daniil-lyakhov daniil-lyakhov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!
It would be great if you would separate the small typo fixes from the min-max implementation, it's too much small changes for me (and probably for others as well)

nncf/experimental/torch2/engine.py Outdated Show resolved Hide resolved
nncf/experimental/torch2/model_transformer.py Show resolved Hide resolved
tests/torch2/utils.py Show resolved Hide resolved
@daniil-lyakhov
Copy link
Collaborator

Are conformance tests numbers available?

@AlexanderDokuchaev
Copy link
Collaborator Author

Model metric FQ int8 metric (N) FQ (N) int8 (N)
torchvision/resnet18 0.6945 30 21 0.69488 30 21
timm/mobilenetv3_small_050 0.4261 62 36 0.4192 62 36
timm/deit3_small_patch16_224 0.8126 74 50 0.8126 74 50
timm/crossvit_9_240 0.7275 112 88 0.72746 112 88
hf/bert-base-uncased   74 77   74 77

(N) - experimental tracing without F/BC

alexsu52
alexsu52 reviewed Jan 27, 2025
View reviewed changes
Copy link
Contributor

@alexsu52 alexsu52 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any performance numbers (speed and memory consumption) for t2f tracing approach? I believe that the value of the PR will be higher if you add it to the PR description.

nncf/experimental/torch2/function_hook/nncf_graph/nncf_graph_builder.py Outdated Show resolved Hide resolved
nncf/quantization/quantize_model.py Outdated Show resolved Hide resolved
nncf/quantization/algorithms/min_max/torch_backend.py Show resolved Hide resolved
nncf/quantization/algorithms/min_max/torch_backend.py Outdated Show resolved Hide resolved
nncf/quantization/algorithms/min_max/torch_backend.py Show resolved Hide resolved
nncf/torch/quantization/layers.py Outdated Show resolved Hide resolved
nncf/torch/quantization/layers.py Outdated Show resolved Hide resolved
nncf/torch/quantization/layers.py Outdated Show resolved Hide resolved
nncf/experimental/torch2/commands.py Show resolved Hide resolved
nncf/experimental/torch2/engine.py Outdated Show resolved Hide resolved
@AlexanderDokuchaev
Copy link
Collaborator Author

Model MinMax. time (develop) ←PR Stat. col. time (develop) ←PR RAM MiB (develop) ←PR
bert-base-uncased 0:00:56 0:00:52 0:00:53 0:00:49 535 553
resnet18 0:00:14 0:00:14 0:00:12 0:00:13 1297 1359
crossvit_9_240 0:01:03 0:00:56 0:00:59 0:00:53 1477 1579
deit3_small_patch16_224 0:00:40 0:00:38 0:00:37 0:00:36 1746 1854
mobilenetv3_small_050 0:00:24 0:00:24 0:00:22 0:00:22 1231 1221

alexsu52
alexsu52 reviewed Jan 28, 2025
View reviewed changes
nncf/quantization/quantize_model.py Outdated Show resolved Hide resolved
nncf/common/factory.py Show resolved Hide resolved
alexsu52
alexsu52 approved these changes Jan 29, 2025
View reviewed changes
Copy link
Contributor

@alexsu52 alexsu52 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

daniil-lyakhov
daniil-lyakhov reviewed Jan 29, 2025
View reviewed changes
Copy link
Collaborator

@daniil-lyakhov daniil-lyakhov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor

nncf/experimental/torch2/commands.py Outdated Show resolved Hide resolved
nncf/experimental/torch2/model_transformer.py Outdated Show resolved Hide resolved
nncf/experimental/torch2/quantization/quantize_model.py Outdated Show resolved Hide resolved
nncf/quantization/algorithms/min_max/torch_backend.py Show resolved Hide resolved
tests/torch2/function_hook/quantization/test_quantized_graphs.py Show resolved Hide resolved
@github-actions github-actions bot added the NNCF OpenVINO Pull requests that updates NNCF OpenVINO label Jan 29, 2025
@alexsu52 alexsu52 merged commit 5ad9bc4 into openvinotoolkit:develop Jan 29, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daniil-lyakhov daniil-lyakhov daniil-lyakhov approved these changes

@alexsu52 alexsu52 alexsu52 approved these changes

Assignees
No one assigned
Labels
API Public API-impacting changes experimental NNCF Common Pull request that updates NNCF Common NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PT Pull requests that updates NNCF PyTorch NNCF PTQ Pull requests that updates NNCF PTQ
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@AlexanderDokuchaev @daniil-lyakhov @alexsu52