Skip to content

Commit

Permalink
update bucket size
Browse files Browse the repository at this point in the history
  • Loading branch information
mwitiderrick committed Jun 7, 2023
1 parent 20ede17 commit 286a204
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions examples/aws-text-benchmarks/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Note: DeepSparse uses static input shapes. Since the distribution of inputs for
we can use bucketing where we compile DeepSparse with multiple input shapes and dynamically route inputs.
In the case of `ag_news` (the example dataset in this case), the distribution of token lengths looks like the following:
![Histogram](image.png)

As such, we used buckets of length 64, 128, and 256. DeepSparse runs best with sequence lengths that are multiples of 16.

## Dense Model GPU
Expand Down

0 comments on commit 286a204

Please sign in to comment.