[Design Discussion] Zk Accel API via extra function parameter [Don't merge]

[mratsim]

A followup on #12 and #13.

The changes required were surprisingly reasonable. No lifetime issues, no type pollution, no Send+Sync requirements.

Only the prover side has been updated to use Zal as on the verifier it's supposedly cheap and also there is a parallel iteration part when batch verification that would be incompatible with ZAL backend that use thead-local sxtorage:

halo2/halo2_proofs/src/plonk/verifier/batch.rs

Lines 107 to 130 in fb69aa2

	let final_msm = self
	.items
	.into_par_iter()
	.enumerate()
	.map(|(i, item)| {
	let instances: Vec<Vec<_>> = item
	.instances
	.iter()
	.map(|i| i.iter().map(|c| &c[..]).collect())
	.collect();
	let instances: Vec<_> = instances.iter().map(|i| &i[..]).collect();
	let strategy = BatchStrategy::new(params);
	let mut transcript = Blake2bRead::init(&item.proof[..]);
	verify_proof(params, vk, strategy, &instances, &mut transcript).map_err(|e| {
	tracing::debug!("Batch item {} failed verification: {}", i, e);
	e
	})
	})
	.try_fold(
	|| params.empty_msm(),
	|msm, res| res.map(|proof_msm| accumulate_msm(msm, proof_msm)),
	)
	.try_reduce(|| params.empty_msm(), |a, b| Ok(accumulate_msm(a, b)));

In terms of API, users just need to pass engine to create_proof so breakage is minimal on external API and contained in halo2.

cc @CPerezz re our chat a couple hours ago

[CPerezz]

I like this approach a lot more than the prev two.

I thought about moving the engine to the Circuit trait as an associated type. But it would be missing create_proof and verify_proof functions.
Otherwise, if we add it to the prover, we are missing keygen.

Unsure if we need a better abstraction or something. Or we should simply settle for this.
Maybe we can discuss more during the next days.
Also, I highly suggest to start looking at privacy-scaling-explorations#243 to see how this will fit-in and if we should have anything in mind.

[CPerezz]

We were thinking on removing the poly definitions/utilities from this crate and move them to an external one where we can have MV and UV as well as optimizations like parallelism and maybe ZAL then too.

I'll follow-up on this. But would be nice to coordinate.

[mratsim]

Some preliminary benches (using Pasta curves)

[CPerezz]

First round of thoughts from @ed255 and me: https://hackmd.io/F6W1U6X8SLCqTciJt0nfiQ?edit

Happy to discuss more in a meeting or whatever works best