stopAndReturnDebugTraceRecording and parseValidation consumes too many gas

[NIC619]

Have you ensured that all of these are up to date?

• This repo
• Any dependencies (according to the package.json)

What command(s) is the bug in?

forge test

Operating System

None

Describe the bug

The two operations consume a huge amount of gas which results in OOG in tests if the tests have invoked many simulation (even if default Foundry gas limit is as high as 1073741824).

I log the gas used in the test in my forked repo: https://github.com/NIC619/erc4337-validation/pull/2/files

> forge test -vvvv --mt testSimulate
Ran 1 test for test/SimulatorV060.t.sol:SimulatorTest
[PASS] testSimulate() (gas: 122687982)
Logs:
  Gas used by stopAndReturnDebugTraceRecording 48967003
  // Below are operations in `parseValidation`
  Gas used by getEntities 40063
  Gas used by filterDebugTrace 70460661
  Gas used by validateBannedOpcodes 1094331
  Gas used by validateOutOfGas 211015
  Gas used by validateBannedStorageLocations 543206
  Gas used by validateCalls 487053
  Gas used by validateExtOpcodes 410799
  Gas used by validateCreate 226283
  // Total gas used by `parseValidation`: 73523105

These are simple userOps, for normal userOp the gas consumption can get up to

• stopAndReturnDebugTraceRecording: 113205509
• parseValidation: 176500495

[NIC619]

After pausing gas metering for debug trace recording and parseValidation:

> forge test -vvvv --mt testSimulate
Ran 1 test for test/SimulatorV060.t.sol:SimulatorTest
[PASS] testSimulate() (gas: 34762)
Logs:
  Gas used by stopAndReturnDebugTraceRecording 744
  // Below are operations in `parseValidation`
  Gas used by getEntities 0
  Gas used by filterDebugTrace 0
  Gas used by validateBannedOpcodes 0
  Gas used by validateOutOfGas 0
  Gas used by validateBannedStorageLocations 0
  Gas used by validateCalls 0
  Gas used by validateExtOpcodes 0
  Gas used by validateCreate 0
  // Total gas used by `parseValidation`: 1998

[NIC619]

I opened an issue in Foundry and their suggestion pauseMetering can partially solve the OOG problem. I've opened the PR #13 which adds pause/resume gas metering for debug trace recording and parseValidation.

However, the debug trace recording still consumes too much memory especially if one runs all test suites at once which could invoke hundreds of times of simulateUserOp. Based on the suggestions from Foundry team, one needs to lower the threads in the forge test command (forge test --threads X)) to prevent the process being killed due to OOM.

Overall, if the memory consumption problem of debug trace recording can not be solved, teams that want to leverage this validation library will need to:

• set gas_limit to very high in Foundry config (to avoid MemoryOOG error)
• set memory_limit to very high in Foundry config (to avoid MemoryLimitOOG error)
• stop fuzzing/invariant tests (as it consumes even more memory than normal tests)
• based on one's (or CI's) machine spec, choose the number of threads properly (to avoid process being killed due to OOM)