Skip to content

Commit

Permalink
ci: update benchmarks
Browse files Browse the repository at this point in the history
  • Loading branch information
GitHub Actions Bot committed Sep 27, 2023
1 parent 3875dc0 commit c9e256d
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 224 deletions.
112 changes: 0 additions & 112 deletions BENCHMARKS-arm64.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,121 +3,9 @@
## Table of Contents

- [Benchmark Results](#benchmark-results)
- [Sudoku: compile](#sudoku:-compile)
- [Sudoku: prove](#sudoku:-prove)
- [Sudoku: verify](#sudoku:-verify)
- [Sudoku:](#sudoku:)
- [fibonacci: compile](#fibonacci:-compile)
- [fibonacci: prove](#fibonacci:-prove)
- [fibonacci: verify](#fibonacci:-verify)
- [fibonacci:](#fibonacci:)
- [fibonacci large: compile](#fibonacci-large:-compile)
- [fibonacci large: prove](#fibonacci-large:-prove)
- [fibonacci large: verify](#fibonacci-large:-verify)
- [fibonacci large:](#fibonacci-large:)
- [Blake: compile](#blake:-compile)
- [Blake: prove](#blake:-prove)
- [Blake: verify](#blake:-verify)
- [Blake:](#blake:)

## Benchmark Results

### Sudoku: compile

| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` |
|:-------|:--------------------------|:-------------------------------|:----------------------------------|:------------------------------------|:-------------------------------------------|:------------------------------------ |
| | `223.85 us` (✅ **1.00x**) | `1.24 ms` (❌ *5.54x slower*) | `64.79 ms` (❌ *289.43x slower*) | `317.25 ms` (❌ *1417.25x slower*) | `591.76 ms` (❌ *2643.57x slower*) | `578.46 ms` (❌ *2584.15x slower*) |

### Sudoku: prove

| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` |
|:-------|:-----------------------|:---------------------------------|:---------------------------------|:----------------------------------|:-------------------------------------------|:---------------------------------- |
| | `2.85 s` (✅ **1.00x**) | `293.32 ms` (🚀 **9.70x faster**) | `70.62 ms` (🚀 **40.30x faster**) | `106.91 ms` (🚀 **26.62x faster**) | `10.03 ms` (🚀 **283.88x faster**) | `14.09 ms` (🚀 **202.01x faster**) |

### Sudoku: verify

| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` |
|:-------|:-------------------------|:--------------------------------|:-------------------------------|:--------------------------------|:-------------------------------------------|:--------------------------------- |
| | `37.25 ms` (✅ **1.00x**) | `3.03 ms` (🚀 **12.31x faster**) | `7.72 ms` (🚀 **4.82x faster**) | `3.56 ms` (🚀 **10.45x faster**) | `7.39 ms` (🚀 **5.04x faster**) | `1.17 ms` (🚀 **31.71x faster**) |

### Sudoku:

| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` |
|:-------|:-----------------------|:---------------------------------|:----------------------------------|:---------------------------------|:-------------------------------------------|:--------------------------------- |
| | `2.42 s` (✅ **1.00x**) | `297.18 ms` (🚀 **8.14x faster**) | `142.19 ms` (🚀 **17.00x faster**) | `432.98 ms` (🚀 **5.58x faster**) | `609.16 ms` (🚀 **3.97x faster**) | `594.97 ms` (🚀 **4.06x faster**) |

### fibonacci: compile

| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` |
|:-------|:--------------------------------|:--------------------------------|:--------------------------------|:--------------------------------|:-------------------------------- |
| | `22.74 us` (✅ **1.00x**) | `22.74 us` (✅ **1.00x faster**) | `59.59 us` (❌ *2.62x slower*) | `51.55 us` (❌ *2.27x slower*) | `41.43 us` (❌ *1.82x slower*) |

### fibonacci: prove

| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` |
|:-------|:--------------------------------|:---------------------------------|:---------------------------------|:---------------------------------|:--------------------------------- |
| | `321.30 ms` (✅ **1.00x**) | `634.68 ms` (❌ *1.98x slower*) | `300.72 ms` (✅ **1.07x faster**) | `151.88 ms` (🚀 **2.12x faster**) | `157.30 ms` (🚀 **2.04x faster**) |

### fibonacci: verify

| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` |
|:-------|:--------------------------------|:--------------------------------|:-------------------------------|:-------------------------------|:------------------------------- |
| | `19.49 ms` (✅ **1.00x**) | `24.74 ms` (❌ *1.27x slower*) | `3.02 ms` (🚀 **6.46x faster**) | `2.94 ms` (🚀 **6.64x faster**) | `2.94 ms` (🚀 **6.62x faster**) |

### fibonacci:

| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` |
|:-------|:--------------------------------|:---------------------------------|:---------------------------------|:---------------------------------|:--------------------------------- |
| | `319.92 ms` (✅ **1.00x**) | `614.63 ms` (❌ *1.92x slower*) | `303.57 ms` (✅ **1.05x faster**) | `154.65 ms` (🚀 **2.07x faster**) | `160.05 ms` (🚀 **2.00x faster**) |

### fibonacci large: compile

| | `Triton: fibonacci-1000` | `Miden: iter-1000` |
|:-------|:----------------------------------|:-------------------------------- |
| | `23.64 us` (✅ **1.00x**) | `59.56 us` (❌ *2.52x slower*) |

### fibonacci large: prove

| | `Triton: fibonacci-1000` | `Miden: iter-1000` |
|:-------|:----------------------------------|:------------------------------ |
| | `9.49 s` (✅ **1.00x**) | `2.52 s` (🚀 **3.76x faster**) |

### fibonacci large: verify

| | `Triton: fibonacci-1000` | `Miden: iter-1000` |
|:-------|:----------------------------------|:-------------------------------- |
| | `60.00 ms` (✅ **1.00x**) | `3.37 ms` (🚀 **17.78x faster**) |

### fibonacci large:

| | `Triton: fibonacci-1000` | `Miden: iter-1000` |
|:-------|:----------------------------------|:------------------------------ |
| | `9.56 s` (✅ **1.00x**) | `2.55 s` (🚀 **3.74x faster**) |

### Blake: compile

| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` |
|:-------|:--------------------------------------------|:---------------------------------- |
| | `35.49 s` (✅ **1.00x**) | `193.45 s` (❌ *5.45x slower*) |

### Blake: prove

| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` |
|:-------|:--------------------------------------------|:---------------------------------- |
| | `48.56 s` (✅ **1.00x**) | `19.93 s` (🚀 **2.44x faster**) |

### Blake: verify

| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` |
|:-------|:--------------------------------------------|:---------------------------------- |
| | `282.42 ms` (✅ **1.00x**) | `615.47 ms` (❌ *2.18x slower*) |

### Blake:

| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` |
|:-------|:--------------------------------------------|:---------------------------------- |
| | `83.88 s` (✅ **1.00x**) | `218.47 s` (❌ *2.60x slower*) |

---
Made with [criterion-table](https://github.com/nu11ptr/criterion-table)

112 changes: 0 additions & 112 deletions BENCHMARKS-x86.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,121 +3,9 @@
## Table of Contents

- [Benchmark Results](#benchmark-results)
- [Sudoku: compile](#sudoku:-compile)
- [Sudoku: prove](#sudoku:-prove)
- [Sudoku: verify](#sudoku:-verify)
- [Sudoku:](#sudoku:)
- [fibonacci: compile](#fibonacci:-compile)
- [fibonacci: prove](#fibonacci:-prove)
- [fibonacci: verify](#fibonacci:-verify)
- [fibonacci:](#fibonacci:)
- [fibonacci large: compile](#fibonacci-large:-compile)
- [fibonacci large: prove](#fibonacci-large:-prove)
- [fibonacci large: verify](#fibonacci-large:-verify)
- [fibonacci large:](#fibonacci-large:)
- [Blake: compile](#blake:-compile)
- [Blake: prove](#blake:-prove)
- [Blake: verify](#blake:-verify)
- [Blake:](#blake:)

## Benchmark Results

### Sudoku: compile

| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` |
|:-------|:--------------------------|:-------------------------------|:----------------------------------|:------------------------------------|:-------------------------------------------|:------------------------------------ |
| | `205.74 us` (✅ **1.00x**) | `1.37 ms` (❌ *6.65x slower*) | `98.20 ms` (❌ *477.31x slower*) | `333.02 ms` (❌ *1618.65x slower*) | `671.45 ms` (❌ *3263.56x slower*) | `668.92 ms` (❌ *3251.26x slower*) |

### Sudoku: prove

| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` |
|:-------|:-----------------------|:----------------------------------|:---------------------------------|:----------------------------------|:-------------------------------------------|:--------------------------------- |
| | `4.33 s` (✅ **1.00x**) | `281.03 ms` (🚀 **15.42x faster**) | `96.56 ms` (🚀 **44.88x faster**) | `118.78 ms` (🚀 **36.48x faster**) | `73.67 ms` (🚀 **58.83x faster**) | `84.74 ms` (🚀 **51.14x faster**) |

### Sudoku: verify

| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` |
|:-------|:-------------------------|:--------------------------------|:-------------------------------|:-------------------------------|:-------------------------------------------|:--------------------------------- |
| | `43.14 ms` (✅ **1.00x**) | `2.43 ms` (🚀 **17.78x faster**) | `7.20 ms` (🚀 **5.99x faster**) | `4.42 ms` (🚀 **9.77x faster**) | `7.77 ms` (🚀 **5.55x faster**) | `3.99 ms` (🚀 **10.81x faster**) |

### Sudoku:

| | `Triton` | `Miden` | `Plonk: 3 by 3` | `Halo: 3 by 3` | `Vamp-IR zk-garage plonk: sudoku` | `Vamp-IR halo2: sudoku` |
|:-------|:-----------------------|:----------------------------------|:----------------------------------|:---------------------------------|:-------------------------------------------|:--------------------------------- |
| | `3.74 s` (✅ **1.00x**) | `285.03 ms` (🚀 **13.11x faster**) | `203.30 ms` (🚀 **18.39x faster**) | `461.41 ms` (🚀 **8.10x faster**) | `750.65 ms` (🚀 **4.98x faster**) | `756.46 ms` (🚀 **4.94x faster**) |

### fibonacci: compile

| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` |
|:-------|:--------------------------------|:--------------------------------|:--------------------------------|:--------------------------------|:-------------------------------- |
| | `19.83 us` (✅ **1.00x**) | `19.81 us` (✅ **1.00x faster**) | `66.05 us` (❌ *3.33x slower*) | `56.78 us` (❌ *2.86x slower*) | `45.71 us` (❌ *2.30x slower*) |

### fibonacci: prove

| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` |
|:-------|:--------------------------------|:---------------------------------|:---------------------------------|:---------------------------------|:--------------------------------- |
| | `468.15 ms` (✅ **1.00x**) | `958.73 ms` (❌ *2.05x slower*) | `282.52 ms` (✅ **1.66x faster**) | `136.05 ms` (🚀 **3.44x faster**) | `137.18 ms` (🚀 **3.41x faster**) |

### fibonacci: verify

| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` |
|:-------|:--------------------------------|:--------------------------------|:-------------------------------|:-------------------------------|:------------------------------- |
| | `23.20 ms` (✅ **1.00x**) | `28.89 ms` (❌ *1.25x slower*) | `2.44 ms` (🚀 **9.51x faster**) | `2.38 ms` (🚀 **9.74x faster**) | `2.38 ms` (🚀 **9.73x faster**) |

### fibonacci:

| | `Triton: fibonacci-50` | `Triton: fibonacci-93` | `Miden: iter-93` | `Miden: fixed-92` | `Miden: fixed-50` |
|:-------|:--------------------------------|:---------------------------------|:---------------------------------|:---------------------------------|:--------------------------------- |
| | `462.22 ms` (✅ **1.00x**) | `917.38 ms` (❌ *1.98x slower*) | `284.82 ms` (✅ **1.62x faster**) | `138.46 ms` (🚀 **3.34x faster**) | `139.64 ms` (🚀 **3.31x faster**) |

### fibonacci large: compile

| | `Triton: fibonacci-1000` | `Miden: iter-1000` |
|:-------|:----------------------------------|:-------------------------------- |
| | `19.93 us` (✅ **1.00x**) | `65.96 us` (❌ *3.31x slower*) |

### fibonacci large: prove

| | `Triton: fibonacci-1000` | `Miden: iter-1000` |
|:-------|:----------------------------------|:------------------------------ |
| | `14.66 s` (✅ **1.00x**) | `2.59 s` (🚀 **5.66x faster**) |

### fibonacci large: verify

| | `Triton: fibonacci-1000` | `Miden: iter-1000` |
|:-------|:----------------------------------|:-------------------------------- |
| | `74.79 ms` (✅ **1.00x**) | `2.71 ms` (🚀 **27.58x faster**) |

### fibonacci large:

| | `Triton: fibonacci-1000` | `Miden: iter-1000` |
|:-------|:----------------------------------|:------------------------------ |
| | `14.39 s` (✅ **1.00x**) | `2.59 s` (🚀 **5.55x faster**) |

### Blake: compile

| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` |
|:-------|:--------------------------------------------|:---------------------------------- |
| | `46.79 s` (✅ **1.00x**) | `205.72 s` (❌ *4.40x slower*) |

### Blake: prove

| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` |
|:-------|:--------------------------------------------|:---------------------------------- |
| | `54.54 s` (✅ **1.00x**) | `27.90 s` (🚀 **1.95x faster**) |

### Blake: verify

| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` |
|:-------|:--------------------------------------------|:---------------------------------- |
| | `398.57 ms` (✅ **1.00x**) | `939.26 ms` (❌ *2.36x slower*) |

### Blake:

| | `Vamp-IR zk-garage plonk: Blake2s` | `Vamp-IR halo2: Blake2s` |
|:-------|:--------------------------------------------|:---------------------------------- |
| | `101.01 s` (✅ **1.00x**) | `234.49 s` (❌ *2.32x slower*) |

---
Made with [criterion-table](https://github.com/nu11ptr/criterion-table)

0 comments on commit c9e256d

Please sign in to comment.