Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add batch precompile #989

Merged
merged 23 commits into from
Aug 11, 2023
Merged

feat: add batch precompile #989

merged 23 commits into from
Aug 11, 2023

Conversation

gitofdeepanshu
Copy link
Contributor

@gitofdeepanshu gitofdeepanshu commented Jul 31, 2023

Pull Request Summary

Batch Precompile

The Batch precompile allows users to combine multiple calls into a single transaction. These calls are executed on behalf of the precompile caller. The caller provides a list of addresses to call, along with corresponding values and call data. If the values and call data arrays are shorter than the list of addresses, default values (0 and empty data) are used for the missing indices.

There are three available functions to determine how the precompile behaves in case of call failures:

BatchSome: If a subcall fails, it reverts that specific subcall but continues to execute any remaining calls in the batch.
BatchSomeUntilFailure: If a subcall fails, it reverts that specific subcall and stops executing any following subcalls. However, the precompile itself does not revert.
BatchAll: If any subcall fails, the precompile reverts itself, undoing any previous successful calls in the batch as well.

Check list

  • added or updated unit tests
  • updated Astar official documentation
  • added OnRuntimeUpgrade hook for precompile revert code registration
  • updated spec version
  • updated semver

@gitofdeepanshu gitofdeepanshu added the runtime This PR/Issue is related to the topic “runtime”. label Aug 2, 2023
precompiles/utils/Cargo.toml Show resolved Hide resolved
precompiles/batch/src/lib.rs Outdated Show resolved Hide resolved
precompiles/batch/Batch.sol Show resolved Hide resolved
precompiles/batch/src/lib.rs Outdated Show resolved Hide resolved
precompiles/batch/src/lib.rs Outdated Show resolved Hide resolved
@gitofdeepanshu gitofdeepanshu marked this pull request as ready for review August 6, 2023 06:20
runtime/shibuya/src/precompiles.rs Show resolved Hide resolved
runtime/local/Cargo.toml Show resolved Hide resolved
runtime/shibuya/Cargo.toml Show resolved Hide resolved
Copy link
Member

@Dinonard Dinonard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. Some minor comments.

precompiles/batch/Batch.sol Outdated Show resolved Hide resolved
precompiles/utils/Cargo.toml Show resolved Hide resolved
/// Move the reading cursor with provided length, and return a range from the previous cursor
/// location to the new one.
/// Checks cursor overflows.
fn move_cursor(&mut self, len: usize) -> EvmResult<Range<usize>> {
pub fn move_cursor(&mut self, len: usize) -> EvmResult<Range<usize>> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be public - we discussed it either here already or in another PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the exact code that was used from moonbeam.
they don't have to make it public because they have defined this function in mod.rs which makes this function public inside a module so the don't have to explicitly make it public which is not the case for us.

https://github.com/moonbeam-foundation/moonbeam/blob/c784d124ead5112859b2e1f813f96d4fd6274ebd/precompiles/utils/src/solidity/codec/bytes.rs#L103

The best we can do is to make a wrapper around this function but then we will have to make that public so it will be the same thing, if you have a better way then kindly suggest.

Copy link
Member

@Dinonard Dinonard Aug 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we also discussed the nature of this issue before.
This is why you added pub fn get_input_from_range, right?

If you need a functionality which is not available in the existing interface, the answer is not to just make the encapsulated internals public. That's bad design.

Since pub fn get_input_from_range is not good enough for your needs, adapt it so it is.

Since I took a more detailed look now, isn't the existing fn read_raw_bytes exactly what you need?

    pub fn read_raw_bytes(&mut self, len: usize) -> EvmResult<&[u8]> {
        let range = self.move_cursor(len)?;

        let data = self
            .input
            .get(range)
            .ok_or_else(|| revert("tried to parse raw bytes out of bounds"))?;

        Ok(data)
    }

compared to yours:

    /// Return Option<&[u8]> from a given range for EvmDataReader
    pub fn get_input_from_range(&self, range: Range<usize>) -> Option<&[u8]> {
        self.input.get(range)
    }

...

impl<K: Kind, S: Get<u32>> EvmData for BoundedBytesString<K, S> {
    fn read(reader: &mut EvmDataReader) -> EvmResult<Self> {
       ...
        // Get valid range over the bytes data.
        let range = inner_reader.move_cursor(array_size)?;

        let data = inner_reader
            .get_input_from_range(range)
            .ok_or_else(|| revert(K::signature()))?;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you are right, we can use this function but I would still argue that this just happens to be a wrapper function that is already public and doesn't add to the improvement of design.

But we can have discussion over this later, I have changed this as per your suggestion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The difference is that move_cursor manipulates internal fields/attributes.
read_raw_bytes is a higher level function that performs a more complex operation, and hides implementation details.

Maybe not the best read, but first thing Google throws out:
https://doc.rust-lang.org/book/ch17-01-what-is-oo.html#encapsulation-that-hides-implementation-details

The less functionality you expose to the outside, the better.
The higher level the exposed functionality is, the better.
Plus, all the stuff mentioned before (changing existing private to public).

Internet is full of resources to read up about this.

@@ -285,7 +290,7 @@ impl EvmDataWriter {

/// Write arbitrary bytes.
/// Doesn't handle any alignement checks, prefer using `write` instead if possible.
fn write_raw_bytes(mut self, value: &[u8]) -> Self {
pub fn write_raw_bytes(mut self, value: &[u8]) -> Self {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The testing module brings a lot of new features - should the existing precompiles be updated to use them in their tests?
(not as part of this PR of course)

Or is this just the split of old testing.rs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it brings some new features but it is backward compatible.

precompiles/batch/src/lib.rs Outdated Show resolved Hide resolved
}
}

Ok(succeed(EvmDataWriter::new().write(true).build()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

God function! 🙂

I understand it's code reuse, but try checking the UT coverage, just in case.
There are a lot of branches to account for.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been designed carefully, we only allow it to pass for the function like BatchSome or BatchSomeUntilFailure and not for BatchAll.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sure it was but still, it's an incredibly complex function - is it fully covered by UTs?
You can use tarpaulin tool to check this: cargo tarpaulin -o html
(it's not a perfect tool but gives good enough picture)

We are just reusing this, we hadn't really written it from scratch. It's easy to overlook something when just reading/reusing the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran tarpaulin and I think it is a good score.
precompiles/batch/src/lib.rs: 127/143

I checked all the tests again, we have enough tests imo, MB has done good testing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't know to say whether that's a good score or not.

But checking coverage in %, and even better, taking a look at the output file, gives a clearer picture if some important function/branch was missed.

Copy link
Member

@shaunxw shaunxw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Some nitpicks.

precompiles/batch/Batch.sol Outdated Show resolved Hide resolved
precompiles/batch/Batch.sol Outdated Show resolved Hide resolved
precompiles/batch/Cargo.toml Outdated Show resolved Hide resolved
precompiles/utils/src/lib.rs Outdated Show resolved Hide resolved
Dinonard
Dinonard previously approved these changes Aug 11, 2023
Copy link
Member

@Dinonard Dinonard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ashutoshvarma
ashutoshvarma previously approved these changes Aug 11, 2023
@github-actions
Copy link

Code Coverage

Package Line Rate Branch Rate Health
pallets/dapps-staking/src 81% 0%
pallets/pallet-xcm/src 53% 0%
chain-extensions/types/dapps-staking/src 0% 0%
pallets/block-reward/src 85% 0%
chain-extensions/pallet-assets/src 0% 0%
chain-extensions/types/assets/src 0% 0%
chain-extensions/dapps-staking/src 0% 0%
pallets/collator-selection/src 69% 0%
pallets/contracts-migration/src 0% 0%
pallets/dapps-staking/src/pallet 85% 0%
pallets/ethereum-checked/src 48% 0%
pallets/xvm/src 46% 0%
precompiles/assets-erc20/src 76% 0%
precompiles/batch/src 80% 0%
precompiles/substrate-ecdsa/src 78% 0%
precompiles/utils/src/testing 62% 0%
pallets/custom-signatures/src 51% 0%
chain-extensions/types/xvm/src 0% 0%
pallets/xc-asset-config/src 54% 0%
precompiles/dapps-staking/src 93% 0%
chain-extensions/xvm/src 0% 0%
precompiles/utils/macro/src 0% 0%
precompiles/utils/src 68% 0%
precompiles/xvm/src 83% 0%
primitives/src 69% 0%
precompiles/xcm/src 84% 0%
precompiles/sr25519/src 79% 0%
primitives/src/xcm 67% 0%
Summary 57% (2567 / 4467) 0% (0 / 0)

Minimum allowed line rate is 50%

@ashutoshvarma ashutoshvarma merged commit c0d3999 into master Aug 11, 2023
12 checks passed
@ashutoshvarma ashutoshvarma deleted the feat/batch-precompile branch August 11, 2023 12:37
PierreOssun pushed a commit that referenced this pull request Aug 15, 2023
* feat: add batch precompile

* typo

* add tests

* minor fixes and updates

* refactor, add batch to local

* fix expect_arguments

* add logs

* update license

* taplo fix

* more refactor, remote evm crate

* remove evm crate, update utils version, add std in dappsstaking

* update pallet_balances::config in mock

* update pallet_evm config

* update license

* add to std-feature

* utils: rectify  patch to minor update

* remove pub from logs function

* fix typos

* remove comments

* refactor

* fmt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
runtime This PR/Issue is related to the topic “runtime”.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants