[Wait for #2568][Mixed] Mixed Precision Layer update #2579
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We will add Var32 Tensor if the Variable Weight is not Full precision (FP32). This eables the Weight Update with full precision and only Apply Gradient Process ueses this Tensor. Therefore, the lifespan of this tensor should be "ApplyGradient". . Modify TensorPool to generate Weigth considering Mixed Precsion. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
This pr create the variable fp32 tensor when we create the Weight and Optimizer Weight. . update the manager to create Weight with var32 tensor which requested to weight pool. . update the weight requests with Weight Spec and var, grad and var32 tensors which created already. . add clone Tensor with specific type in tensor.h Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
This PR enables the FP16 support for the layers below: . input layer . mse loss layer Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
This PR includes the mixed precision test case. . Input - FC - MSE : "batch_size=2", "model_tensor_type=FP16-FP16", "loss_scale=128" **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
This commit modify apply gradient in optimizer. We do not need to save optimizer variables in weight type. Only Optimizer needs the optimizer variables and we should update the weight with full precision to maintain the accuracy. Therefore, remove the var32 tensors for optimizer variables. Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
DonghakPark
requested review from
myungjoo,
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
helloahn,
kparichay,
gichan-jang,
anyj0527 and
zhoonit
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2579. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
DonghakPark
requested review from
lhs8928,
songgot,
jihochu,
SeoHyungjun,
baek2sm,
skykongkong8,
djeong20,
EunjuYang and
a team
as code owners
|
|
cibot: 
|
|
Edited build instructions for Resnet18 test **Fixing the meson build option** Resolves: Error on building the test example where it says `-c is an un-recognized option` and in the meson documentation -C is used, so it seems to be a typo. **Self evaluation:** 1. Build test: []Passed [ ]Failed [ X]Skipped 2. Run test: []Passed [ ]Failed [ X]Skipped Signed-off-by: Udit Jain <udit.jain@samsung.com>
- Implement 4x4 GEMM kernel that works f16-f32 partial accumulation **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <ss.kong@samsung.com>
- Now Hgemm supports 4x4 f16-f32 partial accumulation strategy **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <ss.kong@samsung.com>
- With macro-defined code, the function latency is expected to be optimized by compiler more easily **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <ss.kong@samsung.com>
…16 kernel - With more digits computed with fp16 (in this case 1024 -> 2048) I could observe latency improvement with the cost of accuracy loss. However, according to current accuracy measurement criteria, it is still acceptable. Note that it is highly desired to be proven with model output once more. - With variety of partial sum kernels, we can adaptively apply internal macro kernels without being constrained to K-divisibilty w.r.t. 4, 8, 16.Commit title (Until 50 colums per line) **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <ss.kong@samsung.com>
…8x8 kernel - Apply similar change made in commit#52a3c734 but in 8x8 kernel **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <ss.kong@samsung.com>
- To avoid the constraint of 4-8 divisibilty w.r.t. K, loop for adaptive K direction. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <ss.kong@samsung.com>
- I found there was a repeated usage of matrix initialization before mul-add fused operations. - With separate initialization code, we can enjoy: 1. Cleaner code that is reusable for both f16 & f16-f32 kernel 2. Redundant init process is minimized for f16 kernel. Better latency with the SAME accuracy. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <ss.kong@samsung.com>
- Due to adaptive macro kernel usage, previous comment is no longer needed. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <ss.kong@samsung.com>
Added naive version of OpenCl implementation for FC Layer. Incorporated separate kernels for ops used. Added unit test for fc_layer_cl. Signed-off-by: Debadri Samaddar <s.debadri@samsung.com>
Added incremental forwarding as an option for unit testing layers Signed-off-by: Debadri Samaddar <s.debadri@samsung.com>
Added blas_kernels to enhance resuability of the common blas kernels. Used FullyConnected interface for both CPU and GPU calls. Signed-off-by: Debadri Samaddar <s.debadri@samsung.com>
Renamed global variables in unittest_layers_fully_connected_cl.cpp to fix duplicate declaration error Signed-off-by: Debadri Samaddar <s.debadri@samsung.com>
Fixed kernel argument bug for dot_cl kernel Signed-off-by: Debadri Samaddar <s.debadri@samsung.com>
Used proper size while creating OpenCL buffers. Optimized SGEMM kernel with 2D global work size. Modified function docs. Signed-off-by: Debadri Samaddar <s.debadri@samsung.com>
update yolo v2 modeling part of yolo v2. (update some hyper param values) - update yolo v2 pytorch(python) script - update yolo v2 nntrainer(c++) script * issue - activation function(in this case, leaky relu) of nntrainer needs to support setting negative slope via parameter... **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <sb92.hong@samsung.com>
Added code stub to generate Swiglu layer's golden test data. Signed-off-by: Debadri Samaddar <s.debadri@samsung.com>
It adds tests for conv2d fp16 test. Signed-off-by: Jiho Chu <jiho.chu@samsung.com>
It is assumed that activations and weight are fully compotaible, so it's unnecessary to be converted to. input layer and loss layres are different, cause input data and label data is assumed to be always float 32 type now. Signed-off-by: Jiho Chu <jiho.chu@samsung.com>
This PR is to update the mixed precision layer. - integrate nnstreamer#2568 & nnstreamer#2455 - will update more test **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghak PARK <donghak.park@samsung.com>
taos-ci
approved these changes
May 27, 2024
There was a problem hiding this comment.
@DonghakPark, 💯 All CI checkers are successfully verified. Thanks.
|
will update layers with new PR. |
myungjoo
pushed a commit
that referenced
this pull request
Jun 10, 2024
This PR update the conv2D Layer to support Mixed Precision (FP16). It is based on the PR #2579 Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
DonghakPark
pushed a commit
to DonghakPark/nntrainer
that referenced
this pull request
Oct 8, 2024
This PR update the conv2D Layer to support Mixed Precision (FP16). It is based on the PR nnstreamer#2579 Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
jijoongmoon
added a commit
that referenced
this pull request
Nov 11, 2024
This PR update the conv2D Layer to support Mixed Precision (FP16). It is based on the PR #2579 Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
sugsugsug
pushed a commit
to 2024-2-CID-TEAM-A/nntrainer
that referenced
this pull request
Nov 21, 2024
This PR update the conv2D Layer to support Mixed Precision (FP16). It is based on the PR nnstreamer#2579 Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <jijoong.moon@samsung.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is to update the mixed precision layer.
Self evaluation:
Signed-off-by: Donghak PARK donghak.park@samsung.com