Skip to content

Commit

Permalink
[SPARK-48958][BUILD] Upgrade zstd-jni to 1.5.6-4
Browse files Browse the repository at this point in the history 
### What changes were proposed in this pull request?
The pr aims to upgrade `zstd-jni` from `1.5.6-3` to `1.5.6-4`.

### Why are the changes needed?
1.v1.5.6-3 VS v1.5.6-4
luben/zstd-jni@v1.5.6-3...v1.5.6-4

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Pass GA.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#47432 from panbingkun/SPARK-48958.

Authored-by: panbingkun <panbingkun@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
@panbingkun @dongjoon-hyun
panbingkun authored and dongjoon-hyun committed Jul 22, 2024
1 parent c17fc5a commit bc7a5c0
Show file tree
Hide file tree
Showing 4 changed files with 58 additions and 58 deletions.
56 changes: 28 additions & 28 deletions core/benchmarks/ZStandardBenchmark-jdk21-results.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,48 +2,48 @@
Benchmark ZStandardCompressionCodec
================================================================================================

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1018-azure
OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
Compression 10000 times at level 1 without buffer pool 649 748 156 0.0 64921.9 1.0X
Compression 10000 times at level 2 without buffer pool 689 689 0 0.0 68927.1 0.9X
Compression 10000 times at level 3 without buffer pool 782 782 0 0.0 78180.6 0.8X
Compression 10000 times at level 1 with buffer pool 580 582 2 0.0 57976.0 1.1X
Compression 10000 times at level 2 with buffer pool 614 618 4 0.0 61395.3 1.1X
Compression 10000 times at level 3 with buffer pool 725 734 11 0.0 72535.5 0.9X
Compression 10000 times at level 1 without buffer pool 654 675 20 0.0 65380.3 1.0X
Compression 10000 times at level 2 without buffer pool 714 715 1 0.0 71445.4 0.9X
Compression 10000 times at level 3 without buffer pool 811 814 3 0.0 81142.7 0.8X
Compression 10000 times at level 1 with buffer pool 605 606 1 0.0 60518.7 1.1X
Compression 10000 times at level 2 with buffer pool 634 637 2 0.0 63441.4 1.0X
Compression 10000 times at level 3 with buffer pool 743 743 0 0.0 74258.7 0.9X

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1018-azure
OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
Decompression 10000 times from level 1 without buffer pool 831 832 1 0.0 83114.9 1.0X
Decompression 10000 times from level 2 without buffer pool 834 835 1 0.0 83372.7 1.0X
Decompression 10000 times from level 3 without buffer pool 831 832 1 0.0 83092.3 1.0X
Decompression 10000 times from level 1 with buffer pool 759 760 1 0.0 75870.2 1.1X
Decompression 10000 times from level 2 with buffer pool 759 760 1 0.0 75877.3 1.1X
Decompression 10000 times from level 3 with buffer pool 759 759 0 0.0 75874.5 1.1X
Decompression 10000 times from level 1 without buffer pool 824 826 4 0.0 82358.5 1.0X
Decompression 10000 times from level 2 without buffer pool 824 824 0 0.0 82394.3 1.0X
Decompression 10000 times from level 3 without buffer pool 823 824 1 0.0 82343.3 1.0X
Decompression 10000 times from level 1 with buffer pool 748 749 1 0.0 74792.0 1.1X
Decompression 10000 times from level 2 with buffer pool 748 749 2 0.0 74773.6 1.1X
Decompression 10000 times from level 3 with buffer pool 749 749 1 0.0 74868.3 1.1X

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1018-azure
OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 3: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Parallel Compression with 0 workers 48 50 1 0.0 376632.9 1.0X
Parallel Compression with 1 workers 35 37 2 0.0 272066.6 1.4X
Parallel Compression with 2 workers 34 38 2 0.0 263055.3 1.4X
Parallel Compression with 4 workers 37 39 2 0.0 286835.7 1.3X
Parallel Compression with 8 workers 38 40 1 0.0 299961.3 1.3X
Parallel Compression with 16 workers 43 45 1 0.0 335272.5 1.1X
Parallel Compression with 0 workers 48 49 1 0.0 377356.2 1.0X
Parallel Compression with 1 workers 36 37 2 0.0 279079.1 1.4X
Parallel Compression with 2 workers 36 38 1 0.0 283760.8 1.3X
Parallel Compression with 4 workers 38 40 2 0.0 298581.6 1.3X
Parallel Compression with 8 workers 41 43 1 0.0 320669.0 1.2X
Parallel Compression with 16 workers 46 48 1 0.0 356997.0 1.1X

OpenJDK 64-Bit Server VM 21.0.3+9-LTS on Linux 6.5.0-1018-azure
OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 9: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Parallel Compression with 0 workers 157 158 1 0.0 1224138.2 1.0X
Parallel Compression with 1 workers 187 188 1 0.0 1463264.4 0.8X
Parallel Compression with 2 workers 111 115 6 0.0 863722.6 1.4X
Parallel Compression with 4 workers 105 109 2 0.0 822422.6 1.5X
Parallel Compression with 8 workers 110 114 2 0.0 862852.1 1.4X
Parallel Compression with 16 workers 111 115 2 0.0 870311.3 1.4X
Parallel Compression with 0 workers 159 161 1 0.0 1242020.8 1.0X
Parallel Compression with 1 workers 187 188 1 0.0 1463507.3 0.8X
Parallel Compression with 2 workers 114 118 5 0.0 888481.5 1.4X
Parallel Compression with 4 workers 107 110 2 0.0 836926.1 1.5X
Parallel Compression with 8 workers 110 115 3 0.0 856838.0 1.4X
Parallel Compression with 16 workers 112 115 2 0.0 874554.2 1.4X


56 changes: 28 additions & 28 deletions core/benchmarks/ZStandardBenchmark-results.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,48 +2,48 @@
Benchmark ZStandardCompressionCodec
================================================================================================

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1018-azure
OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
Compression 10000 times at level 1 without buffer pool 656 660 3 0.0 65632.5 1.0X
Compression 10000 times at level 2 without buffer pool 695 696 1 0.0 69509.7 0.9X
Compression 10000 times at level 3 without buffer pool 803 807 7 0.0 80258.4 0.8X
Compression 10000 times at level 1 with buffer pool 584 586 2 0.0 58381.4 1.1X
Compression 10000 times at level 2 with buffer pool 615 616 1 0.0 61463.0 1.1X
Compression 10000 times at level 3 with buffer pool 743 743 0 0.0 74310.9 0.9X
Compression 10000 times at level 1 without buffer pool 259 260 1 0.0 25854.0 1.0X
Compression 10000 times at level 2 without buffer pool 678 678 1 0.0 67756.2 0.4X
Compression 10000 times at level 3 without buffer pool 775 775 1 0.0 77452.6 0.3X
Compression 10000 times at level 1 with buffer pool 572 573 1 0.0 57218.3 0.5X
Compression 10000 times at level 2 with buffer pool 600 602 1 0.0 60041.6 0.4X
Compression 10000 times at level 3 with buffer pool 710 711 2 0.0 70967.2 0.4X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1018-azure
OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
Decompression 10000 times from level 1 without buffer pool 620 621 1 0.0 61972.9 1.0X
Decompression 10000 times from level 2 without buffer pool 622 622 1 0.0 62168.8 1.0X
Decompression 10000 times from level 3 without buffer pool 621 622 1 0.0 62130.0 1.0X
Decompression 10000 times from level 1 with buffer pool 549 550 0 0.0 54939.0 1.1X
Decompression 10000 times from level 2 with buffer pool 550 550 0 0.0 54963.5 1.1X
Decompression 10000 times from level 3 with buffer pool 549 550 1 0.0 54927.7 1.1X
Decompression 10000 times from level 1 without buffer pool 588 588 1 0.0 58764.1 1.0X
Decompression 10000 times from level 2 without buffer pool 589 595 10 0.0 58919.5 1.0X
Decompression 10000 times from level 3 without buffer pool 591 597 12 0.0 59059.2 1.0X
Decompression 10000 times from level 1 with buffer pool 520 521 1 0.0 52039.6 1.1X
Decompression 10000 times from level 2 with buffer pool 521 522 1 0.0 52116.4 1.1X
Decompression 10000 times from level 3 with buffer pool 521 521 0 0.0 52100.6 1.1X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1018-azure
OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 3: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Parallel Compression with 0 workers 47 48 1 0.0 365666.1 1.0X
Parallel Compression with 1 workers 34 36 3 0.0 268562.3 1.4X
Parallel Compression with 2 workers 32 35 2 0.0 251265.1 1.5X
Parallel Compression with 4 workers 35 38 1 0.0 273574.1 1.3X
Parallel Compression with 8 workers 37 40 1 0.0 288217.8 1.3X
Parallel Compression with 16 workers 42 44 1 0.0 330318.7 1.1X
Parallel Compression with 0 workers 46 48 1 0.0 362926.1 1.0X
Parallel Compression with 1 workers 34 36 4 0.0 265302.5 1.4X
Parallel Compression with 2 workers 32 36 1 0.0 252423.6 1.4X
Parallel Compression with 4 workers 36 38 2 0.0 282974.7 1.3X
Parallel Compression with 8 workers 38 40 1 0.0 298633.6 1.2X
Parallel Compression with 16 workers 42 44 1 0.0 329766.4 1.1X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Linux 6.5.0-1018-azure
OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1023-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 9: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Parallel Compression with 0 workers 155 157 2 0.0 1214057.2 1.0X
Parallel Compression with 1 workers 192 193 2 0.0 1499524.2 0.8X
Parallel Compression with 2 workers 112 119 9 0.0 871848.8 1.4X
Parallel Compression with 4 workers 106 109 2 0.0 830699.8 1.5X
Parallel Compression with 8 workers 111 114 2 0.0 870700.3 1.4X
Parallel Compression with 16 workers 112 114 2 0.0 873315.6 1.4X
Parallel Compression with 0 workers 155 156 1 0.0 1212983.5 1.0X
Parallel Compression with 1 workers 191 195 7 0.0 1492776.4 0.8X
Parallel Compression with 2 workers 110 118 7 0.0 861496.4 1.4X
Parallel Compression with 4 workers 105 108 2 0.0 821249.1 1.5X
Parallel Compression with 8 workers 110 114 3 0.0 856234.4 1.4X
Parallel Compression with 16 workers 110 113 2 0.0 855949.6 1.4X


2 changes: 1 addition & 1 deletion dev/deps/spark-deps-hadoop-3-hive-2.3
View file
Original file line number Diff line number Diff line change
Expand Up @@ -275,4 +275,4 @@ xz/1.9//xz-1.9.jar
zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar
zookeeper-jute/3.9.2//zookeeper-jute-3.9.2.jar
zookeeper/3.9.2//zookeeper-3.9.2.jar
zstd-jni/1.5.6-3//zstd-jni-1.5.6-3.jar
zstd-jni/1.5.6-4//zstd-jni-1.5.6-4.jar
2 changes: 1 addition & 1 deletion pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -801,7 +801,7 @@
<dependency>
<groupId>com.github.luben</groupId>
<artifactId>zstd-jni</artifactId>
<version>1.5.6-3</version>
<version>1.5.6-4</version>
</dependency>
<dependency>
<groupId>com.clearspring.analytics</groupId>
Expand Down

0 comments on commit bc7a5c0

Please sign in to comment.