Problem

The systemtests take too long to execute, and their results are non-deterministic.
In particular, the mempool test cases are problematic.

Proposed Solution

Single Chain for Multiple Test Cases

Previously, each set of test cases started a new chain instance, which significantly increased the total test time.
To optimize this, we should modify the setup so that tests reuse an already running chain whenever possible instead of restarting it each time.

Adjusting the Unconfirmed Cosmos Tx Verification Logic

For reasons not yet fully understood, Cosmos transactions tend to remain in the mempool for a shorter period compared to Ethereum transactions.
As a result, checks that verify whether a Cosmos transaction is still pending often fail.
Therefore, for Cosmos transactions expected to be in a pending state, the verification logic should be relaxed to accept both “pending” and “committed” states as valid outcomes.

Block Time Optimization

Currently, some test cases have an excessively long block time configured.
However, even after increasing the block time, certain test cases still fail non-deterministically.

Therefore, for these non-deterministic test cases, we should try to identify the root cause as much as possible.
If the cause cannot be determined, we should create a separate issue, disable the affected test case, and reduce the block time so that the overall test suite can execute more quickly.

Consideration

Parallel Execution of Test Cases

Running test cases in parallel — as is common in unit or integration tests — would greatly reduce execution time, but this is currently difficult to achieve.
This is because multiple test cases share a single chain instance and access the shared systemtests suite concurrently.

While it might be possible to handle shared fields within the test suite itself using a pool with proper locking and unlocking,
it is not feasible to modify shared resources within the cosmossdk.io/systemtest package.

Therefore, the goal is to optimize test execution without introducing parallelization.