From c9e256dce309b0339e19cc34a5986fa567440db7 Mon Sep 17 00:00:00 2001 From: GitHub Actions Bot <> Date: Wed, 27 Sep 2023 07:18:13 +0000 Subject: [PATCH] ci: update benchmarks --- BENCHMARKS-arm64.md | 112 -------------------------------------------- BENCHMARKS-x86.md | 112 -------------------------------------------- 2 files changed, 224 deletions(-) diff --git a/BENCHMARKS-arm64.md b/BENCHMARKS-arm64.md index fd950361..036de898 100644 --- a/BENCHMARKS-arm64.md +++ b/BENCHMARKS-arm64.md @@ -3,121 +3,9 @@ ## Table of Contents - [Benchmark Results](#benchmark-results) - - [Sudoku: compile](#sudoku:-compile) - - [Sudoku: prove](#sudoku:-prove) - - [Sudoku: verify](#sudoku:-verify) - - [Sudoku:](#sudoku:) - - [fibonacci: compile](#fibonacci:-compile) - - [fibonacci: prove](#fibonacci:-prove) - - [fibonacci: verify](#fibonacci:-verify) - - [fibonacci:](#fibonacci:) - - [fibonacci large: compile](#fibonacci-large:-compile) - - [fibonacci large: prove](#fibonacci-large:-prove) - - [fibonacci large: verify](#fibonacci-large:-verify) - - [fibonacci large:](#fibonacci-large:) - - [Blake: compile](#blake:-compile) - - [Blake: prove](#blake:-prove) - - [Blake: verify](#blake:-verify) - - [Blake:](#blake:) ## Benchmark Results -### Sudoku: compile - -| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` | -|:-------|:--------------------------|:-------------------------------|:----------------------------------|:------------------------------------|:-------------------------------------------|:------------------------------------ | -| | `223.85 us` (✅ **1.00x**) | `1.24 ms` (❌ *5.54x slower*) | `64.79 ms` (❌ *289.43x slower*) | `317.25 ms` (❌ *1417.25x slower*) | `591.76 ms` (❌ *2643.57x slower*) | `578.46 ms` (❌ *2584.15x slower*) | - -### Sudoku: prove - -| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` | -|:-------|:-----------------------|:---------------------------------|:---------------------------------|:----------------------------------|:-------------------------------------------|:---------------------------------- | -| | `2.85 s` (✅ **1.00x**) | `293.32 ms` (🚀 **9.70x faster**) | `70.62 ms` (🚀 **40.30x faster**) | `106.91 ms` (🚀 **26.62x faster**) | `10.03 ms` (🚀 **283.88x faster**) | `14.09 ms` (🚀 **202.01x faster**) | - -### Sudoku: verify - -| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` | -|:-------|:-------------------------|:--------------------------------|:-------------------------------|:--------------------------------|:-------------------------------------------|:--------------------------------- | -| | `37.25 ms` (✅ **1.00x**) | `3.03 ms` (🚀 **12.31x faster**) | `7.72 ms` (🚀 **4.82x faster**) | `3.56 ms` (🚀 **10.45x faster**) | `7.39 ms` (🚀 **5.04x faster**) | `1.17 ms` (🚀 **31.71x faster**) | - -### Sudoku: - -| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` | -|:-------|:-----------------------|:---------------------------------|:----------------------------------|:---------------------------------|:-------------------------------------------|:--------------------------------- | -| | `2.42 s` (✅ **1.00x**) | `297.18 ms` (🚀 **8.14x faster**) | `142.19 ms` (🚀 **17.00x faster**) | `432.98 ms` (🚀 **5.58x faster**) | `609.16 ms` (🚀 **3.97x faster**) | `594.97 ms` (🚀 **4.06x faster**) | - -### fibonacci: compile - -| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` | -|:-------|:--------------------------------|:--------------------------------|:--------------------------------|:--------------------------------|:-------------------------------- | -| | `22.74 us` (✅ **1.00x**) | `22.74 us` (✅ **1.00x faster**) | `59.59 us` (❌ *2.62x slower*) | `51.55 us` (❌ *2.27x slower*) | `41.43 us` (❌ *1.82x slower*) | - -### fibonacci: prove - -| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` | -|:-------|:--------------------------------|:---------------------------------|:---------------------------------|:---------------------------------|:--------------------------------- | -| | `321.30 ms` (✅ **1.00x**) | `634.68 ms` (❌ *1.98x slower*) | `300.72 ms` (✅ **1.07x faster**) | `151.88 ms` (🚀 **2.12x faster**) | `157.30 ms` (🚀 **2.04x faster**) | - -### fibonacci: verify - -| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` | -|:-------|:--------------------------------|:--------------------------------|:-------------------------------|:-------------------------------|:------------------------------- | -| | `19.49 ms` (✅ **1.00x**) | `24.74 ms` (❌ *1.27x slower*) | `3.02 ms` (🚀 **6.46x faster**) | `2.94 ms` (🚀 **6.64x faster**) | `2.94 ms` (🚀 **6.62x faster**) | - -### fibonacci: - -| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` | -|:-------|:--------------------------------|:---------------------------------|:---------------------------------|:---------------------------------|:--------------------------------- | -| | `319.92 ms` (✅ **1.00x**) | `614.63 ms` (❌ *1.92x slower*) | `303.57 ms` (✅ **1.05x faster**) | `154.65 ms` (🚀 **2.07x faster**) | `160.05 ms` (🚀 **2.00x faster**) | - -### fibonacci large: compile - -| | `Triton: fibonacci-1000` | `Miden: iter-1000` | -|:-------|:----------------------------------|:-------------------------------- | -| | `23.64 us` (✅ **1.00x**) | `59.56 us` (❌ *2.52x slower*) | - -### fibonacci large: prove - -| | `Triton: fibonacci-1000` | `Miden: iter-1000` | -|:-------|:----------------------------------|:------------------------------ | -| | `9.49 s` (✅ **1.00x**) | `2.52 s` (🚀 **3.76x faster**) | - -### fibonacci large: verify - -| | `Triton: fibonacci-1000` | `Miden: iter-1000` | -|:-------|:----------------------------------|:-------------------------------- | -| | `60.00 ms` (✅ **1.00x**) | `3.37 ms` (🚀 **17.78x faster**) | - -### fibonacci large: - -| | `Triton: fibonacci-1000` | `Miden: iter-1000` | -|:-------|:----------------------------------|:------------------------------ | -| | `9.56 s` (✅ **1.00x**) | `2.55 s` (🚀 **3.74x faster**) | - -### Blake: compile - -| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` | -|:-------|:--------------------------------------------|:---------------------------------- | -| | `35.49 s` (✅ **1.00x**) | `193.45 s` (❌ *5.45x slower*) | - -### Blake: prove - -| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` | -|:-------|:--------------------------------------------|:---------------------------------- | -| | `48.56 s` (✅ **1.00x**) | `19.93 s` (🚀 **2.44x faster**) | - -### Blake: verify - -| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` | -|:-------|:--------------------------------------------|:---------------------------------- | -| | `282.42 ms` (✅ **1.00x**) | `615.47 ms` (❌ *2.18x slower*) | - -### Blake: - -| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` | -|:-------|:--------------------------------------------|:---------------------------------- | -| | `83.88 s` (✅ **1.00x**) | `218.47 s` (❌ *2.60x slower*) | - --- Made with [criterion-table](https://github.com/nu11ptr/criterion-table) diff --git a/BENCHMARKS-x86.md b/BENCHMARKS-x86.md index 2f7b092a..036de898 100644 --- a/BENCHMARKS-x86.md +++ b/BENCHMARKS-x86.md @@ -3,121 +3,9 @@ ## Table of Contents - [Benchmark Results](#benchmark-results) - - [Sudoku: compile](#sudoku:-compile) - - [Sudoku: prove](#sudoku:-prove) - - [Sudoku: verify](#sudoku:-verify) - - [Sudoku:](#sudoku:) - - [fibonacci: compile](#fibonacci:-compile) - - [fibonacci: prove](#fibonacci:-prove) - - [fibonacci: verify](#fibonacci:-verify) - - [fibonacci:](#fibonacci:) - - [fibonacci large: compile](#fibonacci-large:-compile) - - [fibonacci large: prove](#fibonacci-large:-prove) - - [fibonacci large: verify](#fibonacci-large:-verify) - - [fibonacci large:](#fibonacci-large:) - - [Blake: compile](#blake:-compile) - - [Blake: prove](#blake:-prove) - - [Blake: verify](#blake:-verify) - - [Blake:](#blake:) ## Benchmark Results -### Sudoku: compile - -| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` | -|:-------|:--------------------------|:-------------------------------|:----------------------------------|:------------------------------------|:-------------------------------------------|:------------------------------------ | -| | `205.74 us` (✅ **1.00x**) | `1.37 ms` (❌ *6.65x slower*) | `98.20 ms` (❌ *477.31x slower*) | `333.02 ms` (❌ *1618.65x slower*) | `671.45 ms` (❌ *3263.56x slower*) | `668.92 ms` (❌ *3251.26x slower*) | - -### Sudoku: prove - -| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` | -|:-------|:-----------------------|:----------------------------------|:---------------------------------|:----------------------------------|:-------------------------------------------|:--------------------------------- | -| | `4.33 s` (✅ **1.00x**) | `281.03 ms` (🚀 **15.42x faster**) | `96.56 ms` (🚀 **44.88x faster**) | `118.78 ms` (🚀 **36.48x faster**) | `73.67 ms` (🚀 **58.83x faster**) | `84.74 ms` (🚀 **51.14x faster**) | - -### Sudoku: verify - -| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` | -|:-------|:-------------------------|:--------------------------------|:-------------------------------|:-------------------------------|:-------------------------------------------|:--------------------------------- | -| | `43.14 ms` (✅ **1.00x**) | `2.43 ms` (🚀 **17.78x faster**) | `7.20 ms` (🚀 **5.99x faster**) | `4.42 ms` (🚀 **9.77x faster**) | `7.77 ms` (🚀 **5.55x faster**) | `3.99 ms` (🚀 **10.81x faster**) | - -### Sudoku: - -| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` | -|:-------|:-----------------------|:----------------------------------|:----------------------------------|:---------------------------------|:-------------------------------------------|:--------------------------------- | -| | `3.74 s` (✅ **1.00x**) | `285.03 ms` (🚀 **13.11x faster**) | `203.30 ms` (🚀 **18.39x faster**) | `461.41 ms` (🚀 **8.10x faster**) | `750.65 ms` (🚀 **4.98x faster**) | `756.46 ms` (🚀 **4.94x faster**) | - -### fibonacci: compile - -| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` | -|:-------|:--------------------------------|:--------------------------------|:--------------------------------|:--------------------------------|:-------------------------------- | -| | `19.83 us` (✅ **1.00x**) | `19.81 us` (✅ **1.00x faster**) | `66.05 us` (❌ *3.33x slower*) | `56.78 us` (❌ *2.86x slower*) | `45.71 us` (❌ *2.30x slower*) | - -### fibonacci: prove - -| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` | -|:-------|:--------------------------------|:---------------------------------|:---------------------------------|:---------------------------------|:--------------------------------- | -| | `468.15 ms` (✅ **1.00x**) | `958.73 ms` (❌ *2.05x slower*) | `282.52 ms` (✅ **1.66x faster**) | `136.05 ms` (🚀 **3.44x faster**) | `137.18 ms` (🚀 **3.41x faster**) | - -### fibonacci: verify - -| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` | -|:-------|:--------------------------------|:--------------------------------|:-------------------------------|:-------------------------------|:------------------------------- | -| | `23.20 ms` (✅ **1.00x**) | `28.89 ms` (❌ *1.25x slower*) | `2.44 ms` (🚀 **9.51x faster**) | `2.38 ms` (🚀 **9.74x faster**) | `2.38 ms` (🚀 **9.73x faster**) | - -### fibonacci: - -| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` | -|:-------|:--------------------------------|:---------------------------------|:---------------------------------|:---------------------------------|:--------------------------------- | -| | `462.22 ms` (✅ **1.00x**) | `917.38 ms` (❌ *1.98x slower*) | `284.82 ms` (✅ **1.62x faster**) | `138.46 ms` (🚀 **3.34x faster**) | `139.64 ms` (🚀 **3.31x faster**) | - -### fibonacci large: compile - -| | `Triton: fibonacci-1000` | `Miden: iter-1000` | -|:-------|:----------------------------------|:-------------------------------- | -| | `19.93 us` (✅ **1.00x**) | `65.96 us` (❌ *3.31x slower*) | - -### fibonacci large: prove - -| | `Triton: fibonacci-1000` | `Miden: iter-1000` | -|:-------|:----------------------------------|:------------------------------ | -| | `14.66 s` (✅ **1.00x**) | `2.59 s` (🚀 **5.66x faster**) | - -### fibonacci large: verify - -| | `Triton: fibonacci-1000` | `Miden: iter-1000` | -|:-------|:----------------------------------|:-------------------------------- | -| | `74.79 ms` (✅ **1.00x**) | `2.71 ms` (🚀 **27.58x faster**) | - -### fibonacci large: - -| | `Triton: fibonacci-1000` | `Miden: iter-1000` | -|:-------|:----------------------------------|:------------------------------ | -| | `14.39 s` (✅ **1.00x**) | `2.59 s` (🚀 **5.55x faster**) | - -### Blake: compile - -| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` | -|:-------|:--------------------------------------------|:---------------------------------- | -| | `46.79 s` (✅ **1.00x**) | `205.72 s` (❌ *4.40x slower*) | - -### Blake: prove - -| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` | -|:-------|:--------------------------------------------|:---------------------------------- | -| | `54.54 s` (✅ **1.00x**) | `27.90 s` (🚀 **1.95x faster**) | - -### Blake: verify - -| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` | -|:-------|:--------------------------------------------|:---------------------------------- | -| | `398.57 ms` (✅ **1.00x**) | `939.26 ms` (❌ *2.36x slower*) | - -### Blake: - -| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` | -|:-------|:--------------------------------------------|:---------------------------------- | -| | `101.01 s` (✅ **1.00x**) | `234.49 s` (❌ *2.32x slower*) | - --- Made with [criterion-table](https://github.com/nu11ptr/criterion-table)