Regarding better deployment feature #402

[aathan]

Regarding #402:

Some thoughts: Foundry's "solidity first" design principle is awesome for reducing context switching and in this context it has the benefit of promoting parity with other projects which, because they are not solidity based, are by definition both physically and conceptually external to the EVM and have more freedoms to interact with external state.

The features implemented in response to #402 go further towards breaking "the 4th wall" (to borrow a term from film), where the Forge testing code is now orchestrating mainnet transactions with irreversible consequences. It made me extremely nervous to read about the possibility of pointing such code at a mainnet, when its activity may be based on cached values or "copied" state about that mainnet, where those caches may be out of sync with reality. I recently reported (and the team awesomely fixed) various such caching bugs in the build system. For simple (related) reasons, I'd probably only ever use such features if the deployment code in question contained my own explicit checks of chain state in order to decide for itself which deployment steps to continue, retry, omit, etc. And even then, I would carefully --dry-run them first.

Given Foundry's solidity-first approach: When writing tests and likely when using the features discussed in 402, one's mindset is an EVM mindset. You're in some external function that is executing on the basis of some virtual transaction -- albeit one generated by the testing framework and not on-chain (at least insofar as "this" contract). The context is nevertheless transactional.

However, such a transactional context and these features stop short of something which @wighawag touched on: Namely, that without sacrificing the beauty of Foundry's solidity-first approach, we could consider testing of contracts to be part of an "orchestrated" environment where multiple actors, both on and off chain, are participating in tests.

In such an orchestrated context, @wighawag's suggestion to be able to call "pre" and "post" scripts to tests can be cast as a special case of off-chain orchestration. Do X. Then run testA. Then do Y. etc. We could think of "test" functions more like snippets of solidity that can be orchestrated together.

A crude version of this would be a bash script that called forge test -m testA did something and then called forge test -m testB.

The motivations for doing something like that relate to my prior inquiries about gaining access to "external" artifacts (such as logs/events) inside Foundry's solidity-first tests. Ultimately, I acquiesced to the response that vm.warp(); etc should be enough. After all, those are the limits of what a contract can see of the external world.

However, consider contracts that interact with off-chain oracles, and which (for counter-MEV reasons) break up certain behaviors across unrelated transactions; and where such inputs may even come from unrelated chains, or from components that you'd like to nudge to take some next step in the testing process before you continue with another testing step; perhaps even only in order to observe a UX update, based on log emissions.

I mean, obviously, supporting such complex tests can get extremely complex and may be beyond the scope of any near term goals for Foundry; but directionally speaking, cheatcodes that allow some form of synchronous message passing and/or conditional test executions and/or access to external state could be used to compose such things. There could be matching forge commands, RPC endpoints, and or other IPC mechanisms to interact with those cheatcodes. Yes, such message passing could be modeled as normal EVM transactions (e.g., to a Test contract at a well known address). However, not only may that be too heavy, but those interactions may precisely not be appropriate to be recorded on-chain when interacting with mainnet and orchestrating actual transactions (vs tests).

Of course, synchronous pauses or access to externalities within a solidity function breaks the evm/solidity external function execution context paradigm--but then again, so do cheatcodes in general! I.e. it wouldn't necessarily make the most sense to a solidity programmer to see vm.waitFor("step1");doSomething();vm.waitFor("step2");doSomethingElse(); . Therefore, what I've talked about so far (assuming you're not rolling your eyes) maybe only makes sense between such calls, which could therefore be modeled (in order to remain syntax-compatible with solidity) as modifiers to test functions, and as cheatcodes that are only sensical (if not limited to only being valid) within those modifiers.

modifier runOnlyWhenExternalStateGreaterThan(uint barrier)...

Thanks for building Foundry.

[aathan]

PS: I understand that forge is not solc; and that solc does not have pre/post processing features yet ... and that therefore there may not be enough information to know whether cheatcodes are being executed in a modifier or in a body, nor enough hooks to be able to embed execution hints anywhere by as encodings in function names or the like; but maybe you guys are parsing intermediate representations or the like?

[brockelmore]

Interesting thoughts, appreciate them! I do think there is value in allowing to send transactions "in context", i.e. pausing control flow and injecting a return value from something like synchronousBroadcast(). this would likely return a boolean + revert reason strong similar to a low level call in solidity. I think that in general that's a good extension to have but shouldn't be default behavior as the usecase here is much more niche relative to "out of context" (meaning transactions are sent outside of the evm context after all code has been ran) execution

[aathan]

I understand your comments. However, I hadn't internalized that the existing implementation collects all of the broadcasts, only sending them post-transaction (out of context). I guess we're inventing some terminology here.

Irrespective of that issue, and just to highlight/clarify: The current post-transaction broadcast plays directly into what I was saying about orchestrating at the external function level. Solidity as a scripting language is not really great because of the limited data structures, language features etc, but to the extent this is balanced by less context-switching and by the ability to natively and directly pull in on-chain data that's ok.

Put more simply, I can see the next step here being facilities to trigger particular test functions under specific conditions or triggers, including ones that are external to the the executing test contract -- facilitated by cheatcodes that allow for that synchronization & data flow.

[aathan]

Just thinking out loud here but ... wouldn't it be interesting if you could intermix Rust and Solidity (not ethers-rs like) i.e. directly invoke Solidity functions from Rust (or Python, or Lua, or similar) code that has a direct reference to the target test contract(s). I don't mean via client RPCs to an on-chain EVM but rather in a more direct way within whatever execution context Forge itself is calling the tests from.

There must be a test runner class within Forge that sets up the execution context for the tests and then runs them. Making that thing scriptable would be killer.

EDIT: ... err, or maybe I do mean like ethers-rs. I haven't looked at the internals of Forge yet, but I guess it's probably built on top of ethers-rs; so is what I'm thinking about really a matter of either being able to dynamically load rust code to execute in the context of the test running ... or simply scripting of ethers-rs integrated into Forge...