Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverage/fix #459

Merged
merged 4 commits into from
Aug 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .solcover.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,8 @@ module.exports = {
mnemonic: process.env.MNEMONIC,
},
skipFiles: ["test"],
mocha: {
fgrep: "[skip-on-coverage]",
invert: true,
},
};
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@ npm run coverage:mock
Then open the file `coverage/index.html`. You can see there which line or branch for each contract which has been covered or missed by your test suite. This allows increased security by pointing out missing branches not covered yet by the current tests.

> [!Note]
> Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the most important change is the `TFHE.isInitialized` method which will always return `true` in the mocked version. Another big difference in mocked mode, compared to the real fhEVM implementation, is that there is no ciphertext verification neither checking that a ciphertext has been honestly obtained (see section 4 of the [whitepaper](https://github.com/zama-ai/fhevm/blob/main/fhevm-whitepaper.pdf)). This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `npm run test`.
> Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the main difference is the difference in gas prices for the FHE operations. This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `npm run test`.

<p align="right">
<a href="#about" > ↑ Back to top </a>
Expand Down
119 changes: 85 additions & 34 deletions docs/fundamentals/write_contract/hardhat.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,16 @@
# Using Hardhat

This document explains how to start writing smart contract using [Zama Hardhat template](https://github.com/zama-ai/fhevm-hardhat-template).

The Hardhat template allows you to start a fhEVM docker image and run your smart contract on it. Refer to the [README](https://github.com/zama-ai/fhevm-hardhat-template/blob/main/README.md) for more information.

## Developing and testing

When developing confidential contracts, we recommend to use the first the mocked mode of fhEVM for better developer experience:

- For faster testing: `pnpm test:mock`
- For coverage computation: `pnpm coverage:mock`

{% hint style="info" %}
Note that the mocked fhEVM has limitations and discrepancies compared to the real fhEVM node. Refer to the warning section below for details.
{% endhint %}

Ensure to run tests of the final contract version using the real fhEVM. To do this, run `pnpm test` before deployment.
The best way to start writing smart contracts with fhEVM is to use our [Hardhat template](https://github.com/zama-ai/fhevm-hardhat-template).
It allows you to start a fhEVM docker image and run your smart contract on it. Read the [README](https://github.com/zama-ai/fhevm-hardhat-template/blob/main/README.md) for more information.
When developing confidential contracts, we recommend to use first the mocked version of fhEVM for faster testing with `pnpm test:mock` and coverage computation via `pnpm coverage:mock`, this will lead to a better developer experience. However, keep in mind that the mocked fhEVM has some limitations and discrepancies compared to the real fhEVM node, as explained in the warning section at the end of this page.
It's essential to run tests of the final contract version using the real fhEVM. You can do this by running `pnpm test` before deployment.

## Mocked mode

For faster testing iterations, use a mocked version of the `TFHE.sol` library instead of launching all the tests on the local fhEVM node via `pnpm test`or `npx hardhat test`, which could last several minutes.

The same tests should (almost always) pass without any modification: neither the javascript files neither the solidity files need to be changed between the mocked and the real version.
For faster testing iterations, instead of launching all the tests on the local fhEVM node via `pnpm test`or `npx hardhat test` which could last several minutes, you could use instead a mocked version of the fhEVM.
The same tests should (almost always) pass, as is, without any modification: neither the javascript files neither the solidity files need to be changed between the mocked and the real version. The mocked mode does not actually real encryption for encrypted types and runs the tests on a local hardhat node which is implementing the original EVM (i.e non-fhEVM). Additionally, the mocked mode will let you use all the hardhat related special testing/debugging methods, such as `evm_mine`, `evm_snapshot`, `evm_revert` etc, which are very helpful for testing.

The mocked mode does not actually encrypt the encrypted types. It runs the tests on a local hardhat node which is implementing the original EVM (non-fhEVM).

### Running mocked tests

Run the mocked tests using:
To run the mocked tests use either:

```
pnpm test:mock
Expand All @@ -40,10 +23,7 @@ npx hardhat test --network hardhat
```

In mocked mode, all tests should pass in few seconds instead of few minutes, allowing a better developer experience.

### Coverage in mocked mode

Furthermore, getting the coverage of tests is only possible in mocked mode. Use:
Furthermore, getting the coverage of tests is only possible in mocked mode. Just use the following command:

```
pnpm coverage:mock
Expand All @@ -52,13 +32,84 @@ pnpm coverage:mock
Or equivalently:

```
npx hardhat coverage-mock --network hardhat
npx hardhat coverage
```

Then open the file `coverage/index.html`. This increases security by pointing out missing branches not covered yet by the current test suite.

{% hint style="warning" %} Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM. The most important change is that the `TFHE.isInitialized` method always returns `true` in the mocked version.
Then open the file `coverage/index.html` to see the coverage results. This will increase security by pointing out missing branches not covered yet by the current test suite.

**Notice :** Due to limitations in the `solidity-coverage` package, the coverage computation in fhEVM does not support tests involving the `evm_snapshot`hardhat testing method, however, this method is still supported when running tests in mocked mode! In case you are using hardhat snapshots, we recommend you to end your test description by the`[skip-on-coverage]` tag. Here is a concrete example for illustration purpose:

```js
import { expect } from 'chai';
import { ethers, network } from 'hardhat';

import { createInstances, decrypt8, decrypt16, decrypt32, decrypt64 } from '../instance';
import { getSigners, initSigners } from '../signers';
import { deployRandFixture } from './Rand.fixture';

describe('Rand', function () {
before(async function () {
await initSigners();
this.signers = await getSigners();
});

beforeEach(async function () {
const contract = await deployRandFixture();
this.contractAddress = await contract.getAddress();
this.rand = contract;
this.instances = await createInstances(this.signers);
});

it('64 bits generate with upper bound and decrypt', async function () {
const values: bigint[] = [];
for (let i = 0; i < 5; i++) {
const txn = await this.rand.generate64UpperBound(262144);
await txn.wait();
const valueHandle = await this.rand.value64();
const value = await decrypt64(valueHandle);
expect(value).to.be.lessThanOrEqual(262141);
values.push(value);
}
// Expect at least two different generated values.
const unique = new Set(values);
expect(unique.size).to.be.greaterThanOrEqual(2);
});

it('8 and 16 bits generate and decrypt with hardhat snapshots [skip-on-coverage]', async function () {
if (network.name === 'hardhat') {
// snapshots are only possible in hardhat node, i.e in mocked mode
this.snapshotId = await ethers.provider.send('evm_snapshot');
const values: number[] = [];
for (let i = 0; i < 5; i++) {
const txn = await this.rand.generate8();
await txn.wait();
const valueHandle = await this.rand.value8();
const value = await decrypt8(valueHandle);
expect(value).to.be.lessThanOrEqual(0xff);
values.push(value);
}
// Expect at least two different generated values.
const unique = new Set(values);
expect(unique.size).to.be.greaterThanOrEqual(2);

await ethers.provider.send('evm_revert', [this.snapshotId]);
const values2: number[] = [];
for (let i = 0; i < 5; i++) {
const txn = await this.rand.generate8();
await txn.wait();
const valueHandle = await this.rand.value8();
const value = await decrypt8(valueHandle);
expect(value).to.be.lessThanOrEqual(0xff);
values2.push(value);
}
// Expect at least two different generated values.
const unique2 = new Set(values2);
expect(unique2.size).to.be.greaterThanOrEqual(2);
}
});
});
```

Another big difference in mocked mode, compared to the real fhEVM implementation, is that there is no ciphertext verification, neither the checking if a ciphertext has been honestly obtained (see section `4` of the [whitepaper](../../../fhevm-whitepaper.pdf)).
In the previous snippet, the first test will be run in every case, whether in "real" non-mocked mode (`pnpm test`), testing mocked mode (`pnpm test:mock`) or coverage (mocked) mode (`pnpm coverage:mock`). On the other hand, the second test will be run **only** in testing mocked mode, i.e only when running `pnpm test:mock`, since snapshots only works in that specific case. Actually, the second test will be skipped if run in coverage mode, since its description string ends with `[skip-on-coverage]` and similarly, we avoid the test to fail in non-mocked mode since we check that the network name is `hardhat`.

Before deploying to production, you must run the tests with the original fhEVM node as a final check in non-mocked mode, using `pnpm test` or `npx hardhat test`.{% endhint %}
⚠️ **Warning :** Due to intrinsic limitations of the original EVM, the mocked version differ in few corner cases from the real fhEVM, the main difference is the difference in gas prices for the FHE operations. This means that before deploying to production, developers still need to run the tests with the original fhEVM node, as a final check in non-mocked mode, with `pnpm test` or `npx hardhat test`.
24 changes: 7 additions & 17 deletions hardhat.config.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
import '@nomicfoundation/hardhat-toolbox';
import dotenv from 'dotenv';
import * as fs from 'fs';
import 'hardhat-deploy';
import 'hardhat-ignore-warnings';
import type { HardhatUserConfig, extendProvider } from 'hardhat/config';
import { task } from 'hardhat/config';
import type { NetworkUserConfig } from 'hardhat/types';
import { resolve } from 'path';
import * as path from 'path';

import CustomProvider from './CustomProvider';
// Adjust the import path as needed
Expand All @@ -24,19 +22,6 @@ extendProvider(async (provider, config, network) => {
return newProvider;
});

// Function to recursively get all .sol files in a folder
function getAllSolidityFiles(dir: string, fileList: string[] = []): string[] {
fs.readdirSync(dir).forEach((file) => {
const filePath = path.join(dir, file);
if (fs.statSync(filePath).isDirectory()) {
getAllSolidityFiles(filePath, fileList);
} else if (filePath.endsWith('.sol')) {
fileList.push(filePath);
}
});
return fileList;
}

task('compile:specific', 'Compiles only the specified contract')
.addParam('contract', "The contract's path")
.setAction(async ({ contract }, hre) => {
Expand Down Expand Up @@ -89,11 +74,17 @@ function getChainConfig(chain: keyof typeof chainIds): NetworkUserConfig {
};
}

task('coverage').setAction(async (taskArgs, hre, runSuper) => {
hre.config.networks.hardhat.allowUnlimitedContractSize = true;
hre.config.networks.hardhat.blockGasLimit = 1099511627775;

await runSuper(taskArgs);
});

task('test', async (taskArgs, hre, runSuper) => {
// Run modified test task
if (hre.network.name === 'hardhat') {
// in fhevm mode all this block is done when launching the node via `pnmp fhevm:start`
await hre.run('clean');
await hre.run('compile:specific', { contract: 'lib' });

const targetAddress = '0x000000000000000000000000000000000000005d';
Expand Down Expand Up @@ -139,7 +130,6 @@ const config: HardhatUserConfig = {
mnemonic,
path: "m/44'/60'/0'/0",
},
gasPrice: 20000000000, // 20 Gwei
},
zama: getChainConfig('zama'),
localDev: getChainConfig('local'),
Expand Down
Loading