implement Blake2s opcode in runner

[ohad-nir-starkware]

Blake2s opcode runner

Description

Adding the opcode Blake2s to the VM.
Expects op0 to be a pointer to a sequence of 8 felts and and op1 to be a pointer to a sequence of 16 felts.
Said felts should represent u32 integers, i.e. have value of at most 2**32-1.
The 8 felts of op0 represent a state and the 16 felts of op1 represent a message.
dst should hold the value of the counter.
The "output" consists of 8 felts representing u32 numbers of the output of the (non last block) Blake2s compression.
ap should store a pointer, it points to a sequence of 8 cells which each should either be uninitialised or already contain a value matching that of the output at the same index.
The opcode inserts the aforementioned output into the 8 cells [[ap]], [[ap]+1], ... [[ap]+7] (and yields an error if one of said cells already contains a value differing from the output).

Currently Blake2s has opcode_num 8, meaning encoded_instr can use all 64 bits and InstructionNonZeroHighBits is no longer needed as an error.

The motivation is that it has been decided at Starkware that Blake2s is to be implemented at an opcode, thus it needs to be supported by the runner.

Checklist

• Linked to Github Issue
• Unit tests added
• Integration tests added.
• This change requires new documentation.
  • Documentation has been added/updated.
  • CHANGELOG has been updated.

---

This change is 

[github-actions]

**Hyper Thereading Benchmark results**




hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     27.596 s ±  0.001 s    [User: 26.840 s, System: 0.754 s]
  Range (min … max):   27.595 s … 27.596 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     28.070 s ±  0.066 s    [User: 27.321 s, System: 0.747 s]
  Range (min … max):   28.023 s … 28.116 s    2 runs
 
Summary
  hyper_threading_main threads: 1 ran
    1.02 ± 0.00 times faster than hyper_threading_pr threads: 1




hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     15.361 s ±  0.108 s    [User: 26.894 s, System: 0.802 s]
  Range (min … max):   15.285 s … 15.438 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     15.538 s ±  0.089 s    [User: 27.440 s, System: 0.783 s]
  Range (min … max):   15.476 s … 15.601 s    2 runs
 
Summary
  hyper_threading_main threads: 2 ran
    1.01 ± 0.01 times faster than hyper_threading_pr threads: 2




hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     11.418 s ±  0.055 s    [User: 39.268 s, System: 0.963 s]
  Range (min … max):   11.379 s … 11.457 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):     11.085 s ±  0.017 s    [User: 40.866 s, System: 0.958 s]
  Range (min … max):   11.073 s … 11.098 s    2 runs
 
Summary
  hyper_threading_pr threads: 4 ran
    1.03 ± 0.01 times faster than hyper_threading_main threads: 4




hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):     10.713 s ±  0.141 s    [User: 40.296 s, System: 0.923 s]
  Range (min … max):   10.613 s … 10.813 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):     11.350 s ±  0.081 s    [User: 40.935 s, System: 0.931 s]
  Range (min … max):   11.293 s … 11.408 s    2 runs
 
Summary
  hyper_threading_main threads: 6 ran
    1.06 ± 0.02 times faster than hyper_threading_pr threads: 6




hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):     10.842 s ±  0.129 s    [User: 40.010 s, System: 0.984 s]
  Range (min … max):   10.751 s … 10.933 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):     11.149 s ±  0.069 s    [User: 41.087 s, System: 0.993 s]
  Range (min … max):   11.100 s … 11.198 s    2 runs
 
Summary
  hyper_threading_main threads: 8 ran
    1.03 ± 0.01 times faster than hyper_threading_pr threads: 8




hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):     10.850 s ±  0.195 s    [User: 40.412 s, System: 1.072 s]
  Range (min … max):   10.712 s … 10.989 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):     11.159 s ±  0.139 s    [User: 41.376 s, System: 1.066 s]
  Range (min … max):   11.060 s … 11.257 s    2 runs
 
Summary
  hyper_threading_main threads: 16 ran
    1.03 ± 0.02 times faster than hyper_threading_pr threads: 16

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.39%. Comparing base (526285c) to head (4c2fdad).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1927      +/-   ##
==========================================
+ Coverage   96.37%   96.39%   +0.02%     
==========================================
  Files         102      102              
  Lines       41245    41483     +238     
==========================================
+ Hits        39749    39987     +238     
  Misses       1496     1496              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Benchmark Results for unmodified programs 🚀

Command	Mean [s]	Min [s]	Max [s]	Relative
base big_factorial	2.215 ± 0.056	2.182	2.374	1.01 ± 0.03
head big_factorial	2.201 ± 0.018	2.186	2.243	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base big_fibonacci	2.155 ± 0.015	2.140	2.181	1.00
head big_fibonacci	2.164 ± 0.023	2.135	2.214	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base blake2s_integration_benchmark	8.395 ± 0.255	8.027	8.962	1.04 ± 0.03
head blake2s_integration_benchmark	8.053 ± 0.112	7.931	8.262	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base compare_arrays_200000	2.270 ± 0.023	2.235	2.294	1.00
head compare_arrays_200000	2.285 ± 0.036	2.259	2.379	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
base dict_integration_benchmark	1.488 ± 0.009	1.477	1.510	1.00 ± 0.01
head dict_integration_benchmark	1.485 ± 0.011	1.469	1.502	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base field_arithmetic_get_square_benchmark	1.300 ± 0.008	1.288	1.314	1.03 ± 0.01
head field_arithmetic_get_square_benchmark	1.263 ± 0.005	1.256	1.272	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base integration_builtins	8.332 ± 0.135	8.043	8.568	1.03 ± 0.03
head integration_builtins	8.117 ± 0.215	7.947	8.682	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base keccak_integration_benchmark	8.566 ± 0.149	8.331	8.729	1.04 ± 0.02
head keccak_integration_benchmark	8.257 ± 0.052	8.172	8.329	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base linear_search	2.252 ± 0.021	2.225	2.291	1.01 ± 0.01
head linear_search	2.235 ± 0.015	2.203	2.254	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base math_cmp_and_pow_integration_benchmark	1.534 ± 0.005	1.528	1.543	1.00
head math_cmp_and_pow_integration_benchmark	1.548 ± 0.005	1.540	1.555	1.01 ± 0.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base math_integration_benchmark	1.499 ± 0.006	1.486	1.506	1.00
head math_integration_benchmark	1.503 ± 0.015	1.488	1.541	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base memory_integration_benchmark	1.244 ± 0.005	1.238	1.253	1.00 ± 0.01
head memory_integration_benchmark	1.241 ± 0.009	1.221	1.251	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base operations_with_data_structures_benchmarks	1.616 ± 0.004	1.611	1.622	1.00
head operations_with_data_structures_benchmarks	1.630 ± 0.013	1.616	1.653	1.01 ± 0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base pedersen	541.7 ± 4.0	533.5	548.5	1.00 ± 0.01
head pedersen	539.8 ± 3.7	533.3	544.7	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base poseidon_integration_benchmark	640.2 ± 4.7	632.9	647.3	1.00 ± 0.01
head poseidon_integration_benchmark	640.1 ± 6.1	628.4	648.0	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base secp_integration_benchmark	1.854 ± 0.015	1.837	1.876	1.00
head secp_integration_benchmark	1.858 ± 0.012	1.839	1.873	1.00 ± 0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base set_integration_benchmark	606.8 ± 4.2	600.8	614.6	1.00
head set_integration_benchmark	624.5 ± 6.6	615.5	632.7	1.03 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base uint256_integration_benchmark	4.412 ± 0.040	4.376	4.516	1.02 ± 0.01
head uint256_integration_benchmark	4.310 ± 0.041	4.270	4.383	1.00

[DavidLevitGurevich]

I don't understand this sentence

[DavidLevitGurevich]

did you mean said -> same?

[ohad-nir-starkware]

Done.

[DavidLevitGurevich]

just opcode_blake2s or execute_blake2s or blake2s or run_blake2s

[ohad-nir-starkware]

Done.

[Stavbe]

Reviewable status: 0 of 11 files reviewed, 9 unresolved discussions (waiting on @DavidLevitGurevich, @fmoletta, @gabrielbosio, @igaray, @juanbono, @ohad-nir-starkware, @Oppen, @pefontana, and @YairVaknin-starkware)

---

vm/src/vm/vm_core.rs line 1007 at r2 (raw file):

        for (i, x) in res_cow.iter().enumerate() {
            let le_digits = x.clone().into_owned().to_le_digits();
            if le_digits[0] >= (1 << 32)

This conversion to u32 can be moved to a function and also called from handle_blake_2s

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 46 at r2 (raw file):

    assert blake2s_ptr[7] = 0x5BE0CD19;
    static_assert STATE_SIZE_FELTS == 8;
    let blake2s_ptr = blake2s_ptr + STATE_SIZE_FELTS;

Another variable with the same name on purpose?

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 50 at r2 (raw file):

    let (cairo_output) = blake2s_inner{range_check_ptr=range_check_ptr, blake2s_ptr=blake2s_ptr}(data=random_message, n_bytes=INPUT_BLOCK_BYTES+4, counter=COUNTER);

    let relevant_output_start = blake2s_ptr_start+INPUT_BLOCK_FELTS+2+STATE_SIZE_FELTS;

2?

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 128 at r2 (raw file):

    let vm_output_start = cast([ap], felt*);
    return vm_output_start;
}

add new line

---

vm/src/vm/decoding/decoder.rs line 8 at r2 (raw file):

};

//          opcode|ap_update|pc_update|res_logic|op1_src|op0_reg|dst_reg

opcode_extension | opcode |
16 15

---

vm/src/vm/decoding/decoder.rs line 101 at r2 (raw file):

    let opcode_extension = match opcode_extension_num {
        0 => OpcodeExtension::Stone,
        _ => {

1 ?

---

vm/src/vm/decoding/decoder.rs line 102 at r2 (raw file):

        0 => OpcodeExtension::Stone,
        _ => {
            if opcode != Opcode::NOp {

another assertion:
-> op1 src can be only 2/4
res -> op1
pc_update-> regular

[DavidLevitGurevich]

Reviewed 1 of 3 files at r2.
Reviewable status: 1 of 11 files reviewed, 14 unresolved discussions (waiting on @fmoletta, @gabrielbosio, @igaray, @juanbono, @ohad-nir-starkware, @Oppen, @pefontana, @Stavbe, and @YairVaknin-starkware)

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 46 at r2 (raw file):

Previously, Stavbe wrote…

Another variable with the same name on purpose?

this is a standard pattern

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 9 at r2 (raw file):

// Tests the Blake2s opcode runner using a preexisting implementation within the repo as reference.
// The initial state, a random message of 68 bytes and counter are used as input.

why are you using 68 bytes (more than a single block) if you are testing only one block?

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 48 at r2 (raw file):

    let blake2s_ptr = blake2s_ptr + STATE_SIZE_FELTS;

    let (cairo_output) = blake2s_inner{range_check_ptr=range_check_ptr, blake2s_ptr=blake2s_ptr}(data=random_message, n_bytes=INPUT_BLOCK_BYTES+4, counter=COUNTER);

why do you compare to blake2s_inner and not to blake2s_compress?

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 58 at r2 (raw file):

    );

    tempvar check_nonempty = vm_output_start[0];

does it really check that it's nonempty? if you don't initialize it on porpuse does it shout?

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 67 at r2 (raw file):

    tempvar check_nonempty = vm_output_start[7];

    assert vm_output_start[0] = relevant_output_start[0];

you can avoid this by passing the vm_output_start as input to run_blake2s

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 88 at r2 (raw file):

// those denoting uses of fp as the base for operand addresses and flag_opcode_blake (16th flag).
// The instruction is then written to [pc] and the runner is forced to execute Blake2s.
func run_blake2s(

you can have the is_last_block flag as input, place 2 dws and jump to the correct one

[ohad-nir-starkware]

Reviewable status: 1 of 11 files reviewed, 13 unresolved discussions (waiting on @DavidLevitGurevich, @fmoletta, @gabrielbosio, @igaray, @juanbono, @Oppen, @pefontana, @Stavbe, and @YairVaknin-starkware)

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 50 at r2 (raw file):

Previously, Stavbe wrote…

2?

blake2s_inner writes t0 and f0 into the blake_ptr segment

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 58 at r2 (raw file):

Previously, DavidLevitGurevich wrote…

does it really check that it's nonempty? if you don't initialize it on porpuse does it shout?

yes.

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 67 at r2 (raw file):

Previously, DavidLevitGurevich wrote…

you can avoid this by passing the vm_output_start as input to run_blake2s

avoid what?

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 88 at r2 (raw file):

Previously, DavidLevitGurevich wrote…

you can have the is_last_block flag as input, place 2 dws and jump to the correct one

thanks, maybe that's a good idea in the blake_last_block PR (#1932)
does that mean you want both tests to be done in one cairo file?

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 128 at r2 (raw file):

Previously, Stavbe wrote…

add new line

Done.

---

vm/src/vm/decoding/decoder.rs line 8 at r2 (raw file):

Previously, Stavbe wrote…

opcode_extension | opcode |
16 15

Done.

---

vm/src/vm/decoding/decoder.rs line 101 at r2 (raw file):

Previously, Stavbe wrote…

1 ?

Done.

---

vm/src/vm/decoding/decoder.rs line 102 at r2 (raw file):

Previously, Stavbe wrote…

another assertion:
-> op1 src can be only 2/4
res -> op1
pc_update-> regular

Done.

[DavidLevitGurevich]

Reviewed 1 of 4 files at r9, 9 of 9 files at r10, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @fmoletta, @gabrielbosio, @igaray, @juanbono, @Oppen, @pefontana, @Stavbe, and @YairVaknin-starkware)

---

vm/src/vm/decoding/decoder.rs line 426 at r10 (raw file):

    fn decode_opcode_extension_clash() {
        // opcode_extension|   opcode|ap_update|pc_update|res_logic|op1_src|op0_reg|dst_reg
        //  31 ... 17 16 15| 14 13 12|    11 10|  9  8  7|     6  5|4  3  2|      1|      0

Why did you start from 31?

[ohad-nir-starkware]

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @DavidLevitGurevich, @fmoletta, @gabrielbosio, @igaray, @juanbono, @Oppen, @pefontana, and @YairVaknin-starkware)

---

vm/src/vm/decoding/decoder.rs line 426 at r10 (raw file):

Previously, DavidLevitGurevich wrote…

Why did you start from 31?

Done.

[JulianGCalderon]

Hi @ohad-nir-starkware! I left you some test related comments.

[DavidLevitGurevich]

Reviewed 1 of 4 files at r11, 3 of 4 files at r12, 3 of 10 files at r13, 8 of 8 files at r14, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @fmoletta, @gabrielbosio, @igaray, @juanbono, @ohad-nir-starkware, @Oppen, @pefontana, and @YairVaknin-starkware)

---

vm/src/vm/decoding/decoder.rs line 426 at r10 (raw file):

Previously, ohad-nir-starkware (Ohad Nir) wrote…

Done.

this is better. Still not sure why it had to be changed at all, in particular in this PR, but non-blocking

[Stavbe]

Reviewable status: 6 of 13 files reviewed, 4 unresolved discussions (waiting on @DavidLevitGurevich, @fmoletta, @gabrielbosio, @igaray, @juanbono, @ohad-nir-starkware, @Oppen, @pefontana, and @YairVaknin-starkware)

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 8 at r15 (raw file):

from starkware.cairo.common.cairo_builtins import BitwiseBuiltin

const COUNTER = 128;

I think it is better to use the correct value (64)

[ohad-nir-starkware]

Reviewable status: 6 of 13 files reviewed, 4 unresolved discussions (waiting on @DavidLevitGurevich, @fmoletta, @gabrielbosio, @igaray, @juanbono, @JulianGCalderon, @Oppen, @pefontana, @Stavbe, and @YairVaknin-starkware)

---

cairo_programs/stwo_exclusive_programs/blake2s_opcode_test.cairo line 8 at r15 (raw file):

Previously, Stavbe wrote…

I think it is better to use the correct value (64)

Done.

[DavidLevitGurevich]

Reviewed 2 of 4 files at r16.
Reviewable status: 5 of 13 files reviewed, 4 unresolved discussions (waiting on @fmoletta, @gabrielbosio, @igaray, @juanbono, @JulianGCalderon, @Oppen, @pefontana, @Stavbe, and @YairVaknin-starkware)

[ohad-nir-starkware]

Reviewable status: 5 of 13 files reviewed, 4 unresolved discussions (waiting on @DavidLevitGurevich, @fmoletta, @gabrielbosio, @igaray, @juanbono, @JulianGCalderon, @Oppen, @pefontana, @Stavbe, and @YairVaknin-starkware)

[Stavbe]

Reviewable status: 5 of 13 files reviewed, 3 unresolved discussions (waiting on @DavidLevitGurevich, @fmoletta, @gabrielbosio, @igaray, @juanbono, @JulianGCalderon, @Oppen, @pefontana, and @YairVaknin-starkware)

[DavidLevitGurevich]

Reviewed 5 of 7 files at r15, 1 of 4 files at r16, 1 of 8 files at r17.
Reviewable status: 6 of 13 files reviewed, 3 unresolved discussions (waiting on @fmoletta, @gabrielbosio, @igaray, @juanbono, @JulianGCalderon, @Oppen, @pefontana, and @YairVaknin-starkware)

[DavidLevitGurevich]

Reviewed 1 of 8 files at r17.
Reviewable status: 7 of 13 files reviewed, 3 unresolved discussions (waiting on @fmoletta, @gabrielbosio, @igaray, @juanbono, @JulianGCalderon, @Oppen, @pefontana, and @YairVaknin-starkware)

[DavidLevitGurevich]

Reviewed 1 of 8 files at r17, all commit messages.
Reviewable status: 8 of 13 files reviewed, 3 unresolved discussions (waiting on @fmoletta, @gabrielbosio, @igaray, @juanbono, @JulianGCalderon, @Oppen, @pefontana, and @YairVaknin-starkware)

[DavidLevitGurevich]

Reviewed 5 of 8 files at r17.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @fmoletta, @gabrielbosio, @igaray, @juanbono, @JulianGCalderon, @Oppen, @pefontana, and @YairVaknin-starkware)