Execution tracing: GraphQL query to get storage inputs for past blocks #2491
Conversation
Dentosal
added
enhancement
Dentosal
force-pushed
the
dento/execution-trace
branch
from
8d80d7b to
3ba2bc7
Compare
Dentosal
changed the title
Dentosal
changed the title
- Related to #1432 # Release notes In this release, we: - Changed `ABIDecoder` methods to take `std::io::Read` instead of `&[u8]`, allowing it to be used in a streaming manner. # Summary `ABIDecoder` methods take `bytes: impl std::io::Read` instead of `bytes: &[u8]`. This allows decoding abi types without having to know the size in advance. This is particularly useful when reading them directly from VM memory, which will be used by the indexer after FuelLabs/fuel-core#2491 is done. # Breaking Changes `ABIDecoder` methods take `bytes: impl std::io::Read` instead of `bytes: &[u8]`. Callers using arrays or `Vec` must change the argument from `&value` to `value.as_slice()`. # Checklist - [x] All **changes** are **covered** by **tests** (or not applicable) - [x] All **changes** are **documented** (or not applicable) - [x] I **reviewed** the **entire PR** myself (preferably, on GH UI) - [x] I **described** all **Breaking Changes** (or there's none) --------- Co-authored-by: hal3e <git@hal3e.io> Co-authored-by: Ahmed Sagdati <37515857+segfault-magnet@users.noreply.github.com> Co-authored-by: segfault-magnet <ahmed.sagdati.ets@gmail.com>
crates/client/assets/schema.sdl
Outdated
| @@ -762,6 +762,10 @@ type Mutation { | |||
| """ | |||
| dryRun(txs: [HexString!]!, utxoValidation: Boolean, gasPrice: U64, blockHeight: U32): [DryRunTransactionExecutionStatus!]! | |||
| """ | |||
| Get execution trace for an already-executed transaction. | |||
| """ | |||
| storageReadReplay(height: U32!): [StorageReadReplayEvent!]! | |||
There was a problem hiding this comment.
I don't know, maybe @Voxelot knows. But I think it should be just Query since it doesn't modify the internal state and is used on the state at height, so it is deterministic.
| fn get_full_block(&self, height: &BlockHeight) -> StorageResult<Block> { | ||
| let block = self.get_block(height)?; | ||
| let transactions = block | ||
| .transactions() | ||
| .iter() | ||
| .map(|id| self.get_transaction(id).map(|tx| tx.into_owned())) | ||
| .collect::<Result<Vec<_>, _>>()?; | ||
| Ok(block.into_owned().uncompress(transactions)) | ||
| } |
There was a problem hiding this comment.
In the implementation of the BlockProducerDatabase port in fuel-core.
crates/services/upgradable-executor/src/storage_access_recorder.rs
Outdated
Show resolved
Hide resolved
Co-authored-by: Green Baneling <XgreenX9999@gmail.com>
| let output = instance.run(module)?; | ||
|
|
||
| match output { | ||
| ReturnType::ExecutionV0(result) => { | ||
| let _ = convert_from_v0_execution_result(result)?; | ||
| } | ||
| ReturnType::ExecutionV1(result) => { | ||
| let _ = convert_from_v1_execution_result(result)?; | ||
| } | ||
| ReturnType::Validation(result) => { | ||
| let _ = result?; | ||
| } | ||
| } |
There was a problem hiding this comment.
I think for simplicity and performance we can avoid decoding the output type
There was a problem hiding this comment.
Also, maybe for performance we want to avoid deserialization of the output in the instance.run, but it is up to you. Because it required a new method like run_without_result
There was a problem hiding this comment.
I think we still need to ensure the execution was successful, and my understanding is that this is the way to do that. I expect ther performance impact of decoding the type to be neglible anyway.
There was a problem hiding this comment.
Usually you re-execute blocks that already were executed and validated in the past, so the result should be correct, unless storage has different state.
| S: KeyValueInspect, | ||
| { | ||
| pub storage: S, | ||
| pub record: Arc<Mutex<Vec<StorageReadReplayEvent>>>, |
There was a problem hiding this comment.
Instead of Arc<Mutex<_>> I think you could use RefCell. I will make StorageAccessRecorder non Send, but I don't think that we have cases where you need it to be Send. But maybe I'm not right
There was a problem hiding this comment.
We require it to be Send here:
| let output = instance.run(module)?; | ||
|
|
||
| match output { | ||
| ReturnType::ExecutionV0(result) => { | ||
| let _ = convert_from_v0_execution_result(result)?; | ||
| } | ||
| ReturnType::ExecutionV1(result) => { | ||
| let _ = convert_from_v1_execution_result(result)?; | ||
| } | ||
| ReturnType::Validation(result) => { | ||
| let _ = result?; | ||
| } | ||
| } |
There was a problem hiding this comment.
Usually you re-execute blocks that already were executed and validated in the past, so the result should be correct, unless storage has different state.
Description
To support local debugging and execution tracing, we need to add an API that returns the state that VM needs to execute transactions. This is done by recording all database accesses during tx validation, called storage read replay (bikeshedding in progress). This data is then exposed as-is through GraphQL. The API is non-trivial to consume, but a seperate client is proviced, see https://github.com/FuelLabs/execution-trace
The implementation exposes database table names and representations directly. Maintaining backwards compatibility with this could turn out to be quite hard.
Requires
--historical-executionflag to enable, as this is otherwise quite expensive.Follow-ups:
Example query:
Open questions:
Checklist
Before requesting review
After merging, notify other teams
See the VM PR FuelLabs/fuel-vm#881.