feat: pop bench pallet logic implementation

[chungquantin]

Features

• Add a new pop-cli command bench to benchmark pallets & parachain.
• Integrates with frame-benchmarking-cli as a subcommand.
• Run benchmarking on pallets without node binary required. Only runtime needed, inspired by FRAME Omni Bencher.
• Nice to have: Support both benchmarking with chain spec and runtime. FRAME Omni Bencher only support benchmarking with runtime wasm. This addition enhances the user experience by allowing the developers to provide either the chain spec file path or the runtime wasm file.

How to test?

Use the below command in your pop-cli project to benchmark the base_parachain.wasm.

cargo run -- bench pallet --runtime=./crates/pop-cli/test-resources/base_parachain.wasm --pallet=pallet_timestamp --extrinsic= --output=./weights.rs

Or run a command pop bench pallet

Benchmark a pallet or parachain

Usage: pop bench
       pop bench <COMMAND>

Commands:
  pallet  Benchmark the extrinsics of FRAME Pallets
  help    Print this message or the help of the given subcommand(s)

To run benchmarking, builds the parachain runtime and specify the runtime path with --runtime

┌   Pop CLI : Benchmarking your pallets
│
▲  NOTE: the `pop bench pallet` is not yet battle tested - double check the results.
│
▲  NOTE: this may take some time...
│
[2025-02-11T06:04:41Z INFO  pallet_collator_selection::pallet] assembling new collators for new session 0 at #0
[2025-02-11T06:04:41Z INFO  pallet_collator_selection::pallet] assembling new collators for new session 1 at #0
[2025-02-11T06:04:42Z INFO  polkadot_sdk_frame::benchmark::pallet] Loading WASM from file
[2025-02-11T06:04:52Z INFO  polkadot_sdk_frame::benchmark::pallet] [  0 % ] Starting benchmark: pallet_timestamp::set
[2025-02-11T06:04:52Z INFO  polkadot_sdk_frame::benchmark::pallet] [ 50 % ] Starting benchmark: pallet_timestamp::on_finalize
Pallet: "pallet_timestamp", Extrinsic: "set", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: `Timestamp::Now` (r:1 w:1)
Proof: `Timestamp::Now` (`max_values`: Some(1), `max_size`: Some(8), added: 503, mode: `MaxEncodedLen`)
Storage: `Aura::CurrentSlot` (r:1 w:0)
Proof: `Aura::CurrentSlot` (`max_values`: Some(1), `max_size`: Some(8), added: 503, mode: `MaxEncodedLen`)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=      111
              µs

Reads = 2
Writes = 1
Recorded proof Size = 86

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=      111
              µs

Reads = 2
Writes = 1
Recorded proof Size = 86

Pallet: "pallet_timestamp", Extrinsic: "on_finalize", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=       50
              µs

Reads = 0
Writes = 0
Recorded proof Size = 57

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=       50
              µs

Reads = 0
Writes = 0
Recorded proof Size = 57

└  Benchmark completed successfully!

[codecov]

Codecov Report

Attention: Patch coverage is 46.03175% with 34 lines in your changes missing coverage. Please review.

Project coverage is 74.99%. Comparing base (2e03233) to head (53258ac).

Files with missing lines	Patch %	Lines
crates/pop-cli/src/commands/bench/mod.rs	50.87%	18 Missing and 10 partials ⚠️
crates/pop-cli/src/main.rs	0.00%	5 Missing ⚠️
crates/pop-cli/src/commands/mod.rs	0.00%	1 Missing ⚠️

@@            Coverage Diff             @@
##             main     #405      +/-   ##
==========================================
- Coverage   75.11%   74.99%   -0.13%     
==========================================
  Files          63       64       +1     
  Lines       13842    13905      +63     
  Branches    13842    13905      +63     
==========================================
+ Hits        10398    10428      +30     
- Misses       2118     2141      +23     
- Partials     1326     1336      +10     

Files with missing lines	Coverage Δ
crates/pop-cli/src/commands/mod.rs	16.00% <0.00%> (-0.33%)	⬇️
crates/pop-cli/src/main.rs	40.50% <0.00%> (-2.74%)	⬇️
crates/pop-cli/src/commands/bench/mod.rs	50.87% <50.87%> (ø)

... and 1 file with indirect coverage changes

[chungquantin]

Requires to log the messages from frame-benchmarking-cli

[chungquantin]

Takes quite quick to benchmark pallet_timestamp. This optimistically expects the test WASM binary is configured with pallet timestamp (common with most of Substrate runtimes)