Skip to content

Commit

Permalink
docs: updated mocked and coverage docs
Browse files Browse the repository at this point in the history
  • Loading branch information
jatZama committed Aug 9, 2024
1 parent db0e844 commit f48df75
Show file tree
Hide file tree
Showing 4 changed files with 88 additions and 37 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@ npm run coverage:mock
Then open the file `coverage/index.html`. You can see there which line or branch for each contract which has been covered or missed by your test suite. This allows increased security by pointing out missing branches not covered yet by the current tests.

> [!Note]
> Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the most important change is the `TFHE.isInitialized` method which will always return `true` in the mocked version. Another big difference in mocked mode, compared to the real fhEVM implementation, is that there is no ciphertext verification neither checking that a ciphertext has been honestly obtained (see section 4 of the [whitepaper](https://github.com/zama-ai/fhevm/blob/main/fhevm-whitepaper.pdf)). This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `npm run test`.
> Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the main difference is the difference in gas prices for the FHE operations. This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `npm run test`.
<p align="right">
<a href="#about" > ↑ Back to top </a>
Expand Down
119 changes: 85 additions & 34 deletions docs/fundamentals/write_contract/hardhat.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,16 @@
# Using Hardhat

This document explains how to start writing smart contract using [Zama Hardhat template](https://github.com/zama-ai/fhevm-hardhat-template).

The Hardhat template allows you to start a fhEVM docker image and run your smart contract on it. Refer to the [README](https://github.com/zama-ai/fhevm-hardhat-template/blob/main/README.md) for more information.

## Developing and testing

When developing confidential contracts, we recommend to use the first the mocked mode of fhEVM for better developer experience:

- For faster testing: `pnpm test:mock`
- For coverage computation: `pnpm coverage:mock`

{% hint style="info" %}
Note that the mocked fhEVM has limitations and discrepancies compared to the real fhEVM node. Refer to the warning section below for details.
{% endhint %}

Ensure to run tests of the final contract version using the real fhEVM. To do this, run `pnpm test` before deployment.
The best way to start writing smart contracts with fhEVM is to use our [Hardhat template](https://github.com/zama-ai/fhevm-hardhat-template).
It allows you to start a fhEVM docker image and run your smart contract on it. Read the [README](https://github.com/zama-ai/fhevm-hardhat-template/blob/main/README.md) for more information.
When developing confidential contracts, we recommend to use first the mocked version of fhEVM for faster testing with `pnpm test:mock` and coverage computation via `pnpm coverage:mock`, this will lead to a better developer experience. However, keep in mind that the mocked fhEVM has some limitations and discrepancies compared to the real fhEVM node, as explained in the warning section at the end of this page.
It's essential to run tests of the final contract version using the real fhEVM. You can do this by running `pnpm test` before deployment.

## Mocked mode

For faster testing iterations, use a mocked version of the `TFHE.sol` library instead of launching all the tests on the local fhEVM node via `pnpm test`or `npx hardhat test`, which could last several minutes.

The same tests should (almost always) pass without any modification: neither the javascript files neither the solidity files need to be changed between the mocked and the real version.
For faster testing iterations, instead of launching all the tests on the local fhEVM node via `pnpm test`or `npx hardhat test` which could last several minutes, you could use instead a mocked version of the fhEVM.
The same tests should (almost always) pass, as is, without any modification: neither the javascript files neither the solidity files need to be changed between the mocked and the real version. The mocked mode does not actually real encryption for encrypted types and runs the tests on a local hardhat node which is implementing the original EVM (i.e non-fhEVM). Additionally, the mocked mode will let you use all the hardhat related special testing/debugging methods, such as `evm_mine`, `evm_snapshot`, `evm_revert` etc, which are very helpful for testing.

The mocked mode does not actually encrypt the encrypted types. It runs the tests on a local hardhat node which is implementing the original EVM (non-fhEVM).

### Running mocked tests

Run the mocked tests using:
To run the mocked tests use either:

```
pnpm test:mock
Expand All @@ -40,10 +23,7 @@ npx hardhat test --network hardhat
```

In mocked mode, all tests should pass in few seconds instead of few minutes, allowing a better developer experience.

### Coverage in mocked mode

Furthermore, getting the coverage of tests is only possible in mocked mode. Use:
Furthermore, getting the coverage of tests is only possible in mocked mode. Just use the following command:

```
pnpm coverage:mock
Expand All @@ -52,13 +32,84 @@ pnpm coverage:mock
Or equivalently:

```
npx hardhat coverage-mock --network hardhat
npx hardhat coverage
```

Then open the file `coverage/index.html`. This increases security by pointing out missing branches not covered yet by the current test suite.

{% hint style="warning" %} Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM. The most important change is that the `TFHE.isInitialized` method always returns `true` in the mocked version.
Then open the file `coverage/index.html` to see the coverage results. This will increase security by pointing out missing branches not covered yet by the current test suite.

**Notice :** Due to limitations in the solidity-coverage package, the coverage computation in fhEVM does not support tests involving the `evm_snapshot` hardhat testing method, however, this method is still supported when running tests in mocked mode! In case you are using hardhat snapshots, we recommend you to end your test description by the `[skip-on-coverage]` tag. Here is a concrete example for illustration purpose:

```js
import { expect } from 'chai';
import { ethers, network } from 'hardhat';

import { createInstances, decrypt8, decrypt16, decrypt32, decrypt64 } from '../instance';
import { getSigners, initSigners } from '../signers';
import { deployRandFixture } from './Rand.fixture';

describe('Rand', function () {
before(async function () {
await initSigners();
this.signers = await getSigners();
});

beforeEach(async function () {
const contract = await deployRandFixture();
this.contractAddress = await contract.getAddress();
this.rand = contract;
this.instances = await createInstances(this.signers);
});

it('64 bits generate with upper bound and decrypt', async function () {
const values: bigint[] = [];
for (let i = 0; i < 5; i++) {
const txn = await this.rand.generate64UpperBound(262144);
await txn.wait();
const valueHandle = await this.rand.value64();
const value = await decrypt64(valueHandle);
expect(value).to.be.lessThanOrEqual(262141);
values.push(value);
}
// Expect at least two different generated values.
const unique = new Set(values);
expect(unique.size).to.be.greaterThanOrEqual(2);
});

it('8 and 16 bits generate and decrypt with hardhat snapshots [skip-on-coverage]', async function () {
if (network.name === 'hardhat') {
// snapshots are only possible in hardhat node, i.e in mocked mode
this.snapshotId = await ethers.provider.send('evm_snapshot');
const values: number[] = [];
for (let i = 0; i < 5; i++) {
const txn = await this.rand.generate8();
await txn.wait();
const valueHandle = await this.rand.value8();
const value = await decrypt8(valueHandle);
expect(value).to.be.lessThanOrEqual(0xff);
values.push(value);
}
// Expect at least two different generated values.
const unique = new Set(values);
expect(unique.size).to.be.greaterThanOrEqual(2);

await ethers.provider.send('evm_revert', [this.snapshotId]);
const values2: number[] = [];
for (let i = 0; i < 5; i++) {
const txn = await this.rand.generate8();
await txn.wait();
const valueHandle = await this.rand.value8();
const value = await decrypt8(valueHandle);
expect(value).to.be.lessThanOrEqual(0xff);
values2.push(value);
}
// Expect at least two different generated values.
const unique2 = new Set(values2);
expect(unique2.size).to.be.greaterThanOrEqual(2);
}
});
});
```

Another big difference in mocked mode, compared to the real fhEVM implementation, is that there is no ciphertext verification, neither the checking if a ciphertext has been honestly obtained (see section `4` of the [whitepaper](../../../fhevm-whitepaper.pdf)).
In the previous snippet, the first test will be run in every case, whether in "real" non-mocked mode (`pnpm test`), testing mocked mode (`pnpm test:mock`) or coverage (mocked) mode (`pnpm coverage:mock`). On the other hand, the second test will be run **only** in testing mocked mode, i.e only when running `pnpm test:mock`, since snapshots only works in that specific case. Actually, the second test will be skipped if run in coverage mode, since its description string ends with `[skip-on-coverage]` and similarly, we avoid the test to fail in non-mocked mode since we check that the network name is `hardhat`.

Before deploying to production, you must run the tests with the original fhEVM node as a final check in non-mocked mode, using `pnpm test` or `npx hardhat test`.{% endhint %}
⚠️ **Warning :** Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the main difference is the difference in gas prices for the FHE operations. This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `pnpm test` or `npx hardhat test`.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"test:inband": "hardhat test",
"test": "HARDHAT_PARALLEL=1 hardhat test --parallel",
"test:mock": "hardhat test --network hardhat",
"coverage:mock": "hardhat coverage-mock --network hardhat",
"coverage:mock": "hardhat coverage",
"typechain": "cross-env TS_NODE_TRANSPILE_ONLY=true hardhat typechain",
"codegen": "npx ts-node codegen/main.ts && npm run prettier",
"codegen:overloads": "npx ts-node codegen/generateOverloads.ts",
Expand Down
2 changes: 1 addition & 1 deletion test/rand/Rand.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ describe('Rand', function () {
const contract = await deployRandFixture();
this.contractAddress = await contract.getAddress();
this.rand = contract;
this.instances = await createInstances(this.contractAddress, ethers, this.signers);
this.instances = await createInstances(this.signers);
});

it('8 bits generate and decrypt', async function () {
Expand Down

0 comments on commit f48df75

Please sign in to comment.