From f48df75feb163fa1bd02e70b6554c8d29233d970 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Joseph-Andr=C3=A9=20Turk?= Date: Fri, 9 Aug 2024 17:52:27 +0200 Subject: [PATCH] docs: updated mocked and coverage docs --- README.md | 2 +- docs/fundamentals/write_contract/hardhat.md | 119 ++++++++++++++------ package.json | 2 +- test/rand/Rand.ts | 2 +- 4 files changed, 88 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index f23e9e24..558bdef9 100644 --- a/README.md +++ b/README.md @@ -316,7 +316,7 @@ npm run coverage:mock Then open the file `coverage/index.html`. You can see there which line or branch for each contract which has been covered or missed by your test suite. This allows increased security by pointing out missing branches not covered yet by the current tests. > [!Note] -> Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the most important change is the `TFHE.isInitialized` method which will always return `true` in the mocked version. Another big difference in mocked mode, compared to the real fhEVM implementation, is that there is no ciphertext verification neither checking that a ciphertext has been honestly obtained (see section 4 of the [whitepaper](https://github.com/zama-ai/fhevm/blob/main/fhevm-whitepaper.pdf)). This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `npm run test`. +> Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the main difference is the difference in gas prices for the FHE operations. This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `npm run test`.

↑ Back to top diff --git a/docs/fundamentals/write_contract/hardhat.md b/docs/fundamentals/write_contract/hardhat.md index 722b4541..6e3b406d 100644 --- a/docs/fundamentals/write_contract/hardhat.md +++ b/docs/fundamentals/write_contract/hardhat.md @@ -1,33 +1,16 @@ # Using Hardhat -This document explains how to start writing smart contract using [Zama Hardhat template](https://github.com/zama-ai/fhevm-hardhat-template). - -The Hardhat template allows you to start a fhEVM docker image and run your smart contract on it. Refer to the [README](https://github.com/zama-ai/fhevm-hardhat-template/blob/main/README.md) for more information. - -## Developing and testing - -When developing confidential contracts, we recommend to use the first the mocked mode of fhEVM for better developer experience: - -- For faster testing: `pnpm test:mock` -- For coverage computation: `pnpm coverage:mock` - -{% hint style="info" %} -Note that the mocked fhEVM has limitations and discrepancies compared to the real fhEVM node. Refer to the warning section below for details. -{% endhint %} - -Ensure to run tests of the final contract version using the real fhEVM. To do this, run `pnpm test` before deployment. +The best way to start writing smart contracts with fhEVM is to use our [Hardhat template](https://github.com/zama-ai/fhevm-hardhat-template). +It allows you to start a fhEVM docker image and run your smart contract on it. Read the [README](https://github.com/zama-ai/fhevm-hardhat-template/blob/main/README.md) for more information. +When developing confidential contracts, we recommend to use first the mocked version of fhEVM for faster testing with `pnpm test:mock` and coverage computation via `pnpm coverage:mock`, this will lead to a better developer experience. However, keep in mind that the mocked fhEVM has some limitations and discrepancies compared to the real fhEVM node, as explained in the warning section at the end of this page. +It's essential to run tests of the final contract version using the real fhEVM. You can do this by running `pnpm test` before deployment. ## Mocked mode -For faster testing iterations, use a mocked version of the `TFHE.sol` library instead of launching all the tests on the local fhEVM node via `pnpm test`or `npx hardhat test`, which could last several minutes. - -The same tests should (almost always) pass without any modification: neither the javascript files neither the solidity files need to be changed between the mocked and the real version. +For faster testing iterations, instead of launching all the tests on the local fhEVM node via `pnpm test`or `npx hardhat test` which could last several minutes, you could use instead a mocked version of the fhEVM. +The same tests should (almost always) pass, as is, without any modification: neither the javascript files neither the solidity files need to be changed between the mocked and the real version. The mocked mode does not actually real encryption for encrypted types and runs the tests on a local hardhat node which is implementing the original EVM (i.e non-fhEVM). Additionally, the mocked mode will let you use all the hardhat related special testing/debugging methods, such as `evm_mine`, `evm_snapshot`, `evm_revert` etc, which are very helpful for testing. -The mocked mode does not actually encrypt the encrypted types. It runs the tests on a local hardhat node which is implementing the original EVM (non-fhEVM). - -### Running mocked tests - -Run the mocked tests using: +To run the mocked tests use either: ``` pnpm test:mock @@ -40,10 +23,7 @@ npx hardhat test --network hardhat ``` In mocked mode, all tests should pass in few seconds instead of few minutes, allowing a better developer experience. - -### Coverage in mocked mode - -Furthermore, getting the coverage of tests is only possible in mocked mode. Use: +Furthermore, getting the coverage of tests is only possible in mocked mode. Just use the following command: ``` pnpm coverage:mock @@ -52,13 +32,84 @@ pnpm coverage:mock Or equivalently: ``` -npx hardhat coverage-mock --network hardhat +npx hardhat coverage ``` -Then open the file `coverage/index.html`. This increases security by pointing out missing branches not covered yet by the current test suite. - -{% hint style="warning" %} Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM. The most important change is that the `TFHE.isInitialized` method always returns `true` in the mocked version. +Then open the file `coverage/index.html` to see the coverage results. This will increase security by pointing out missing branches not covered yet by the current test suite. + +**Notice :** Due to limitations in the solidity-coverage package, the coverage computation in fhEVM does not support tests involving the `evm_snapshot` hardhat testing method, however, this method is still supported when running tests in mocked mode! In case you are using hardhat snapshots, we recommend you to end your test description by the `[skip-on-coverage]` tag. Here is a concrete example for illustration purpose: + +```js +import { expect } from 'chai'; +import { ethers, network } from 'hardhat'; + +import { createInstances, decrypt8, decrypt16, decrypt32, decrypt64 } from '../instance'; +import { getSigners, initSigners } from '../signers'; +import { deployRandFixture } from './Rand.fixture'; + +describe('Rand', function () { + before(async function () { + await initSigners(); + this.signers = await getSigners(); + }); + + beforeEach(async function () { + const contract = await deployRandFixture(); + this.contractAddress = await contract.getAddress(); + this.rand = contract; + this.instances = await createInstances(this.signers); + }); + + it('64 bits generate with upper bound and decrypt', async function () { + const values: bigint[] = []; + for (let i = 0; i < 5; i++) { + const txn = await this.rand.generate64UpperBound(262144); + await txn.wait(); + const valueHandle = await this.rand.value64(); + const value = await decrypt64(valueHandle); + expect(value).to.be.lessThanOrEqual(262141); + values.push(value); + } + // Expect at least two different generated values. + const unique = new Set(values); + expect(unique.size).to.be.greaterThanOrEqual(2); + }); + + it('8 and 16 bits generate and decrypt with hardhat snapshots [skip-on-coverage]', async function () { + if (network.name === 'hardhat') { + // snapshots are only possible in hardhat node, i.e in mocked mode + this.snapshotId = await ethers.provider.send('evm_snapshot'); + const values: number[] = []; + for (let i = 0; i < 5; i++) { + const txn = await this.rand.generate8(); + await txn.wait(); + const valueHandle = await this.rand.value8(); + const value = await decrypt8(valueHandle); + expect(value).to.be.lessThanOrEqual(0xff); + values.push(value); + } + // Expect at least two different generated values. + const unique = new Set(values); + expect(unique.size).to.be.greaterThanOrEqual(2); + + await ethers.provider.send('evm_revert', [this.snapshotId]); + const values2: number[] = []; + for (let i = 0; i < 5; i++) { + const txn = await this.rand.generate8(); + await txn.wait(); + const valueHandle = await this.rand.value8(); + const value = await decrypt8(valueHandle); + expect(value).to.be.lessThanOrEqual(0xff); + values2.push(value); + } + // Expect at least two different generated values. + const unique2 = new Set(values2); + expect(unique2.size).to.be.greaterThanOrEqual(2); + } + }); +}); +``` -Another big difference in mocked mode, compared to the real fhEVM implementation, is that there is no ciphertext verification, neither the checking if a ciphertext has been honestly obtained (see section `4` of the [whitepaper](../../../fhevm-whitepaper.pdf)). +In the previous snippet, the first test will be run in every case, whether in "real" non-mocked mode (`pnpm test`), testing mocked mode (`pnpm test:mock`) or coverage (mocked) mode (`pnpm coverage:mock`). On the other hand, the second test will be run **only** in testing mocked mode, i.e only when running `pnpm test:mock`, since snapshots only works in that specific case. Actually, the second test will be skipped if run in coverage mode, since its description string ends with `[skip-on-coverage]` and similarly, we avoid the test to fail in non-mocked mode since we check that the network name is `hardhat`. -Before deploying to production, you must run the tests with the original fhEVM node as a final check in non-mocked mode, using `pnpm test` or `npx hardhat test`.{% endhint %} +⚠️ **Warning :** Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the main difference is the difference in gas prices for the FHE operations. This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `pnpm test` or `npx hardhat test`. diff --git a/package.json b/package.json index 71861681..54f7c578 100644 --- a/package.json +++ b/package.json @@ -21,7 +21,7 @@ "test:inband": "hardhat test", "test": "HARDHAT_PARALLEL=1 hardhat test --parallel", "test:mock": "hardhat test --network hardhat", - "coverage:mock": "hardhat coverage-mock --network hardhat", + "coverage:mock": "hardhat coverage", "typechain": "cross-env TS_NODE_TRANSPILE_ONLY=true hardhat typechain", "codegen": "npx ts-node codegen/main.ts && npm run prettier", "codegen:overloads": "npx ts-node codegen/generateOverloads.ts", diff --git a/test/rand/Rand.ts b/test/rand/Rand.ts index fffbf06e..4af90781 100644 --- a/test/rand/Rand.ts +++ b/test/rand/Rand.ts @@ -15,7 +15,7 @@ describe('Rand', function () { const contract = await deployRandFixture(); this.contractAddress = await contract.getAddress(); this.rand = contract; - this.instances = await createInstances(this.contractAddress, ethers, this.signers); + this.instances = await createInstances(this.signers); }); it('8 bits generate and decrypt', async function () {