From 286a2042d3c0d11f714dc9e42e1b77bea220ff37 Mon Sep 17 00:00:00 2001 From: Derrick Mwiti Date: Wed, 7 Jun 2023 19:13:07 +0300 Subject: [PATCH] update bucket size --- examples/aws-text-benchmarks/readme.md | 1 + 1 file changed, 1 insertion(+) diff --git a/examples/aws-text-benchmarks/readme.md b/examples/aws-text-benchmarks/readme.md index 021d1c8d1f..085f33cd5d 100644 --- a/examples/aws-text-benchmarks/readme.md +++ b/examples/aws-text-benchmarks/readme.md @@ -26,6 +26,7 @@ Note: DeepSparse uses static input shapes. Since the distribution of inputs for we can use bucketing where we compile DeepSparse with multiple input shapes and dynamically route inputs. In the case of `ag_news` (the example dataset in this case), the distribution of token lengths looks like the following: ![Histogram](image.png) + As such, we used buckets of length 64, 128, and 256. DeepSparse runs best with sequence lengths that are multiples of 16. ## Dense Model GPU