Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pallet-api/fungibles): benchmark read functions #284

Merged
merged 7 commits into from
Sep 13, 2024
73 changes: 72 additions & 1 deletion pallets/api/src/fungibles/benchmarking.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,10 @@ use frame_support::{
use frame_system::RawOrigin;
use sp_runtime::traits::Zero;

use super::{AccountIdOf, AssetsInstanceOf, AssetsOf, BalanceOf, Call, Config, Pallet, TokenIdOf};
use super::{
AccountIdOf, AssetsInstanceOf, AssetsOf, BalanceOf, Call, Config, Pallet, Read, TokenIdOf,
};
use crate::Read as _;

const SEED: u32 = 1;

Expand Down Expand Up @@ -103,5 +106,73 @@ mod benchmarks {
Ok(())
}

#[benchmark]
peterwht marked this conversation as resolved.
Show resolved Hide resolved
// Storage: `Assets`
fn total_supply() {
#[block]
{
Pallet::<T>::read(Read::TotalSupply(TokenIdOf::<T>::zero()));
}
}
#[benchmark]
// Storage: `AssetAccount`
fn balance_of() {
#[block]
{
Pallet::<T>::read(Read::BalanceOf {
token: TokenIdOf::<T>::zero(),
owner: account("Alice", 0, SEED),
});
}
}
#[benchmark]
// Storage: `Approval`
fn allowance() {
#[block]
{
Pallet::<T>::read(Read::Allowance {
token: TokenIdOf::<T>::zero(),
owner: account("Alice", 0, SEED),
spender: account("Bob", 0, SEED),
});
}
}

#[benchmark]
// Storage: `AssetMetadata`
fn token_name() {
#[block]
{
Pallet::<T>::read(Read::TokenName(TokenIdOf::<T>::zero()));
}
}

#[benchmark]
// Storage: `AssetMetadata`
fn token_symbol() {
#[block]
{
Pallet::<T>::read(Read::TokenSymbol(TokenIdOf::<T>::zero()));
}
}

#[benchmark]
// Storage: `AssetMetadata`
fn token_decimals() {
#[block]
{
Pallet::<T>::read(Read::TokenDecimals(TokenIdOf::<T>::zero()));
}
}

#[benchmark]
// Storage: `Assets`
fn token_exists() {
#[block]
{
Pallet::<T>::read(Read::TokenExists(TokenIdOf::<T>::zero()));
}
}

impl_benchmark_test_suite!(Pallet, crate::mock::new_test_ext(), crate::mock::Test);
}
14 changes: 11 additions & 3 deletions pallets/api/src/fungibles/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -490,9 +490,17 @@ pub mod pallet {
///
/// # Parameters
/// - `request` - The read request.
fn weight(_request: &Self::Read) -> Weight {
// TODO: match on request and return benchmarked weight
T::DbWeight::get().reads(1_u64)
fn weight(request: &Self::Read) -> Weight {
use Read::*;
match request {
peterwht marked this conversation as resolved.
Show resolved Hide resolved
TotalSupply(_) => <T as Config>::WeightInfo::total_supply(),
BalanceOf { .. } => <T as Config>::WeightInfo::balance_of(),
Allowance { .. } => <T as Config>::WeightInfo::allowance(),
TokenName(_) => <T as Config>::WeightInfo::token_name(),
TokenSymbol(_) => <T as Config>::WeightInfo::token_symbol(),
TokenDecimals(_) => <T as Config>::WeightInfo::token_decimals(),
TokenExists(_) => <T as Config>::WeightInfo::token_exists(),
}
}

/// Performs the requested read and returns the result.
Expand Down
97 changes: 97 additions & 0 deletions pallets/api/src/fungibles/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -342,3 +342,100 @@ fn set_metadata_token(
) {
assert_ok!(Assets::set_metadata(signed(owner), token, name, symbol, decimals));
}

mod read_weights {
use frame_support::weights::Weight;

use super::*;
use crate::fungibles::{weights::WeightInfo, Config};

struct ReadWeightInfo {
total_supply: Weight,
balance_of: Weight,
allowance: Weight,
token_name: Weight,
token_symbol: Weight,
token_decimals: Weight,
token_exists: Weight,
}

impl ReadWeightInfo {
fn new() -> Self {
Self {
total_supply: Fungibles::weight(&TotalSupply(TOKEN)),
balance_of: Fungibles::weight(&BalanceOf { token: TOKEN, owner: ALICE }),
allowance: Fungibles::weight(&Allowance {
token: TOKEN,
owner: ALICE,
spender: BOB,
}),
token_name: Fungibles::weight(&TokenName(TOKEN)),
token_symbol: Fungibles::weight(&TokenSymbol(TOKEN)),
token_decimals: Fungibles::weight(&TokenDecimals(TOKEN)),
token_exists: Fungibles::weight(&TokenExists(TOKEN)),
}
}
}

#[test]
fn ensure_read_matches_benchmarks() {
let ReadWeightInfo {
allowance,
balance_of,
token_decimals,
token_name,
token_symbol,
total_supply,
token_exists,
} = ReadWeightInfo::new();

assert_eq!(total_supply, <Test as Config>::WeightInfo::total_supply());
assert_eq!(balance_of, <Test as Config>::WeightInfo::balance_of());
assert_eq!(allowance, <Test as Config>::WeightInfo::allowance());
assert_eq!(token_name, <Test as Config>::WeightInfo::token_name());
assert_eq!(token_symbol, <Test as Config>::WeightInfo::token_symbol());
assert_eq!(token_decimals, <Test as Config>::WeightInfo::token_decimals());
assert_eq!(token_exists, <Test as Config>::WeightInfo::token_exists());
}

// These types read from the `AssetMetadata` storage.
#[test]
fn ensure_asset_metadata_variants_match() {
let ReadWeightInfo { token_decimals, token_name, token_symbol, .. } = ReadWeightInfo::new();

assert_eq!(token_decimals, token_name);
assert_eq!(token_decimals, token_symbol);
}

// These types read from the `Assets` storage.
#[test]
fn ensure_asset_variants_match() {
let ReadWeightInfo { total_supply, token_exists, .. } = ReadWeightInfo::new();

assert_eq!(total_supply, token_exists);
}

// Proof size is based on `MaxEncodedLen`, not hardware.
// This test ensures that the data structure sizes do not change with upgrades.
#[test]
fn ensure_expected_proof_size_does_not_change() {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to double check this, please try on your machine!

./target/release/pop-node \
benchmark \
pallet \
--chain=dev \
--wasm-execution=compiled \
--pallet=fungibles \
--steps=50 \
--repeat=20 \
--json \
--template \
./scripts/pallet-weights-template.hbs \
--output=./pallets/api/src/fungibles/weights.rs \
--extrinsic=

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice if we also assert to the maxencodedlen of the structs from pallet assets maybe? How you did it previously

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see latest commit

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant a simple assert like the following:

		assert_eq!(allowance.proof_size(), Approval::<Balance, Balance>::max_encoded_len() as u64);

But that is incorrect. I'm not so sure whether all the additional test code in your last commit is worth it. Increases the burden to maintain and writing benchmarks is tmo a reason that we don't need to test the results in utter most detail.

cc @evilrobot-01

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. Tbh when I saw your comment last night I thought the same, so had the same misunderstanding.

My view is that if we rely on the accuracy of benchmarks/weights for dispatchables, recreating it for reads seems unnecessary. There is a framework there for benchmarking and getting in-depth results out of it, so its reasonable to rely on it as that is what underpins the ecosystem. Fine for some additional sanity checks tho.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whilst I admire the level of detail, clarity and accuracy of these tests, my only reservation would be whether this would become an expectation moving forward. My concern would be the additional time required to implement for all the other APIs to remain consistent versus the value it brings.

Copy link
Collaborator

@Daanvdplas Daanvdplas Sep 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same, therefore I think it is best to remove.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clear detailed docs and tests though Peter, was able to immediately grasp the detailed workings of it!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was quite tedious to do! Although thankful for the learnings :) The last test that verifies the proof size remains the same will effectively provide the same testing outcome (make sure the max_encoded_len does not change) please see the latest commit, reverting the changes.

let ReadWeightInfo {
allowance,
balance_of,
token_decimals,
token_name,
token_symbol,
total_supply,
token_exists,
} = ReadWeightInfo::new();

// These values come from `weights.rs`.
assert_eq!(allowance.proof_size(), 3613);
assert_eq!(balance_of.proof_size(), 3599);
assert_eq!(token_name.proof_size(), 3605);
assert_eq!(token_symbol.proof_size(), 3605);
assert_eq!(token_decimals.proof_size(), 3605);
assert_eq!(total_supply.proof_size(), 3675);
assert_eq!(token_exists.proof_size(), 3675);
}
}
Loading
Loading