Merge pull request #778 from multiversx/development

Development to main

Author: radumojic

docs/developers/data/composite-values.md

@@ -4,7 +4,7 @@ title: Composite Values
 ---
 [comment]: # (mx-abstract)
 
-We often need to group simple values into more complex ones, without splitting them into [several arguments](/developers/data/multi-value).
+We often need to group simple values into more complex ones, without splitting them into [several arguments](/developers/data/multi-values).
 
 Here too we opted for an encoding that is, above all, very easy to read.
 

docs/developers/meta/sc-config.md

@@ -40,12 +40,20 @@ allocator = "leaking"
 stack-size = "3 pages"
 features = ["example_feature_1", "example_feature_2"]
 kill-legacy-callback = true
+
+[contracts.main.profile]
+codegen-units = 1
+opt-level = "z"
+lto = true
+debug = false
+panic = "abort"
+overflow-checks = false
 ```
 
 The settings are as follows:
 - `panic-message`
     - “Panic with message” is a feature very useful for debugging. It displays messages from regular Rust panics in a contract, at the cost of ~1.5kB of additional contract size. It is disabled by default, we advise against using it in production.
-    - _values_: `true` | `false
+    - _values_: `true` | `false`
     - _default_: `false`
 - `ei`
     - Configures the post-processor that checks the environment interface (EI) used by the built smart contract.
@@ -82,7 +90,44 @@ The settings are as follows:
     - _default_: `[]`
 - `kill-legacy-callback`
     - The framework has no way of knowing whether or not a certain smart contract variant actually needs the async callback code or not, so in rare situations it is necessary to forcefully remove it.
-    - _values_: `true` | `false
+    - _values_: `true` | `false`
+    - _default_: `false`
+- `codegen-units`
+    - Controls the number of "code generation units" a crate will be split into. Splitting a crate into multiple code generation units can have a significant impact on the compile time and code optimization of the crate.
+    - From our experience it is of no particular impact to smart contract compilation.
+    - _default_: `1`
+- `opt-level`
+    - Controls the level of optimization that Rust will apply to the code during compilation.
+    - By default we run with `s` or `z`, since for smart contracts bytecode size optimization is of the essence.
+    - We also run `wasm-opt` after this optimization phase, so this only refers to part of the optimization.
+    - _values_:
+        - `0`: no optimizations;
+        - `1`: basic optimizations;
+        - `2`: some optimizations;
+        - `3`: all optimizations;
+        - `s`: optimize for binary size;
+        - `z`: optimize for binary size, but also turn off loop vectorization.
+    - _default_: `z`
+- `lto`
+    - Enables link-time optimization for the release profile.
+    - _values_: `true` | `false`
+    - _default_: `true`
+- `debug`
+    - Controls the amount of debug information included in the compiled binary.
+    - This setting is not normally used, since wasm-opt erases any debug information anyway. Use the `build-dbg` to get debug information for the compiled contracts.
+    - _values_: `true` | `false`
+    - _default_: `false`
+- `panic`
+    - Controls how smart contracts handles panics, which are unexpected errors that occur at runtime.
+    - Using `"unwind"` is not tested as it makes little sense in a smart contract.
+    - _values_:
+        - `"unwind"`: unwind the stack in case of a panic;
+        - `"abort"`: terminate the execution in case of a panic.
+    - _default_: `"abort"`
+- `overflow_checks`
+    - Controls whether it performs runtime checks for integer overflow. When enabled, it will insert additional checks into the generated code to detect and prevent integer overflow errors.
+    - Note that overflow checks are normally turned off in production, but are useful when testing. The overflow checks are enabled by default when testing smart contracts using the debugger.
+    - _values_: `true` | `false`
     - _default_: `false`
 
 
@@ -272,7 +317,7 @@ An _external view contract_ has a behavior different from that of a regular cont
     - A list of endpoint names to be added directly to this contract.
     - It bypasses the label system.
     - Can be useful if for some reason labels are missing in code or deemed too cumbersome.
-    - Use the public endpoin names, not the rust function names.
+    - Use the public endpoint names, not the Rust function names.
     - _values_: a list of endpoint names, e.g. `add-labels = ["myEndpoint1", "myEndpoint1"]`
     - _default_: `[]`
 - `labels-for-contracts`

docs/developers/meta/sc-meta-cli.md

@@ -58,7 +58,7 @@ Paramameters:
 - `--ignore`
     - Ignore all directories with these names.
     - _default_: `target`.
-- `--no-abi-git-version
+- `--no-abi-git-version`
     - Skips loading the Git version into the ABI
 - `--target-dir-meta`
     - For the meta crates, allows specifying the target directory where the Rust compiler will build the intermediary files. Sharing the same target directory can speed up building multiple contract crates at once.
@@ -114,6 +114,9 @@ Paramameters:
 - `--to`
     - Overrides the version to upgrade to.
     - _default_: the last released version.
+- `--no-check`
+    - By default `upgrade` compile checks the project after each major version upgrade. This is to allow developers that upgrade multiple versions to address issues with the upgrade before too many such issues get to accumulate. This feature can be turned off by the `--no-check` flag.
+    - _default_: project is compiled.
 
 [comment]: # (mx-context-auto)
 
@@ -211,7 +214,9 @@ Paramameter:
 
 ### Calling `test-gen`
 
-<!-- TODO: expand section and move to a separate page -->
+The `test-gen` tool is used to [generate boilerplate](/developers/testing/scenario/running-scenarios#auto-generating-the-boilerplate) code when [integrating JSON scenario files in a contract's Rust test suite](/developers/testing/scenario/running-scenarios#integration-in-rust).
+
+In short:
 
 Contracts often have JSON scenario tests associated with them, which normally reside in the `scenarios` folder, under the contract crate root.
 
@@ -221,53 +226,7 @@ These integration tests come in two flavors:
 - Rust tests, that exclusively use the Rust debugger infrastructure;
 - VM tests that use the Go infrastructure.
 
-An example:
-
-```rust title="adder/tests/adder_scenario_rs_test.rs"
-use multiversx_sc_scenario::*;
-
-fn world() -> ScenarioWorld {
-    let mut blockchain = ScenarioWorld::new();
-    blockchain.set_current_dir_from_workspace("contracts/examples/adder");
-
-    blockchain.register_contract("file:output/adder.wasm", adder::ContractBuilder);
-    blockchain
-}
-
-#[test]
-fn adder_rs() {
-    world().run("scenarios/adder.scen.json");
-}
-
-#[test]
-fn interactor_trace_rs() {
-    world().run("scenarios/interactor_trace.scen.json");
-}
-```
-
-```rust title="adder/tests/adder_scenario_go_test.rs"
-use multiversx_sc_scenario::*;
-
-fn world() -> ScenarioWorld {
-    ScenarioWorld::vm_go()
-}
-
-#[test]
-fn adder_go() {
-    world().run("scenarios/adder.scen.json");
-}
-
-#[test]
-fn interactor_trace_go() {
-    world().run("scenarios/interactor_trace.scen.json");
-}
-```
-
-The `world()` definition is expected form the developer, but the tests themselves are generated and updated automatically when calling `sc-meta test-gen`.
-
-:::caution
-The tool does not work well with code that is commented-out. In order to temporarily disable a test, annotate it with `#[ignore]`.
-:::
+Read more about JSON scenarios in smart contract projects [here](/developers/testing/scenario/running-scenarios).
 
 Paramameters:
 - `--path`

docs/developers/overview.md

@@ -143,11 +143,11 @@ the execution of smart contract, information about ESDT transfers or built-in fu
 
 | Name                                                                          | Description                                                                                                                                                                                                         |
 |-------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| [ESDT Operations Events](/developers/log-events/esdt-events)                  | ESDT operations, which encompass token creation, transfers, and other critical actions, generate log events. These events record sender and receiver addresses, token amounts, and operation types.                 |
-| [Execution Events](/developers/log-events/execution-events)                   | Execution events are dedicated to recording the status of transaction execution. They indicate whether a transaction was successfully executed or encountered an error.                                             |
-| [Smart Contract Call Events](/developers/log-events/contract-call-events)     | Smart contract calls often emit log events to report their execution status and results. These events typically include information such as the caller's address, the called function, and any other relevant data. |
-| [Smart Contract Deploy Events](/developers/log-events/contract-deploy-events) | Smart contract deployment and upgrade events are used to record when a smart contract is initially deployed or when it undergoes an upgrade.                                                                        |
-| [System Delegation Events](/developers/log-events/system-delegation-events)   | System delegation events are generated in response to interactions with the system delegation contract.                                                                                                             |
+| [ESDT Operations Events](/developers/event-logs/esdt-events)                  | ESDT operations, which encompass token creation, transfers, and other critical actions, generate log events. These events record sender and receiver addresses, token amounts, and operation types.                 |
+| [Execution Events](/developers/event-logs/execution-events)                   | Execution events are dedicated to recording the status of transaction execution. They indicate whether a transaction was successfully executed or encountered an error.                                             |
+| [Smart Contract Call Events](/developers/event-logs/contract-call-events)     | Smart contract calls often emit log events to report their execution status and results. These events typically include information such as the caller's address, the called function, and any other relevant data. |
+| [Smart Contract Deploy Events](/developers/event-logs/contract-deploy-events) | Smart contract deployment and upgrade events are used to record when a smart contract is initially deployed or when it undergoes an upgrade.                                                                        |
+| [System Delegation Events](/developers/event-logs/system-delegation-events)   | System delegation events are generated in response to interactions with the system delegation contract.                                                                                                             |
 
 
 

docs/developers/scenario-reference/overview.md

@@ -0,0 +1,69 @@
+---
+id: concept
+title: Concept
+---
+
+
+[comment]: # (mx-context-auto)
+
+## What is a scenario?
+
+Let's think for a moment how an interaction with a blockchain might look like.
+
+The only way to change the blockchain state is by sending transctions. We might also need to query some contracts in between sending these transactions. We might also query the blockchain state (balances, for instance) directly. Let's call these actions "steps". In a simulated environment, we need at least an additional step: initializing the sandbox.
+
+Several steps form a scenario.
+
+A scenario is any interaction with a blockchain, composed of one or more steps. These interactions might be:
+- real, completed on a real blockchain, a history of sorts;
+- programmed to be executed in the future;
+- simulated, but using real blockchain data;
+- simulated, using absurd or unrealistic data.
+
+So it doesn't really matter if these steps are real or imagined. All that matters is that they obey the rules of the blockchain.
+
+Because of their generality, it is natural to think of all blockchain interactions and black-box tests as such scenarios.
+
+
+[comment]: # (mx-context-auto)
+
+## Scenario formats
+
+The concept of scenario is not tied to a specific technology, or language.
+
+Historically, they started out as JSON tests. But writing a lot of JSON by hand is very inconvenient and unproductive, so we came up with a very similar syntax in Rust. When we created the interactor framework, we found that the scenario model fits naturally to real interactions too.
+
+:::info important
+Nowadays, we think that the JSON scenario format is best used for interoperability and replays, not for writing tests.
+
+There are several ways to generate a scenario JSON file automatically, and we encourage developers to move away from writing them by hand.
+:::
+
+The greatest benefit of the JSON format is that it is language-agnostic, and so it can be used with any of our backends.
+
+
+[comment]: # (mx-context-auto)
+
+## Scenarios as tests
+
+Scenarios also have syntax for checking transaction outputs and the blockchain state. This means that each scenario is also a test. Any failing check causes the scenario test to fail.
+
+:::important What kind of tests are they?
+They are always **black-box** tests. They model real blockchain interactions, so there is no way for them to peek inside contracts and access their private functions.
+:::
+
+
+[comment]: # (mx-context-auto)
+
+## Typed vs. untyped scenarios
+
+Transaction data on the blockchain is not really typed. Arguments, storage, logs, etc. - they are all untyped at blockchain level. The types are only added by developers when writing contracts, to avoid bugs and make them safer to use.
+
+But this means that scenarios in their most general form are also untyped. This fits JSON well, which is also (mostly) untyped.
+
+It clearly becomes tiring, as a developer, to only be able to work with untyped and un-annotated data. There are two ways to overcome this:
+- Using a typed version of the scenarios. This is what we do when working with scenarios in Rust.
+- Using a specialized language to make values easier to read by developers.
+
+These are the scenario value expressions, and they help us better express numbers, addresses, or even more complex data. Note that this does not add any type checks, it is a purely cosmetic affair. The format is detailed [here](/developers/testing/scenario/values-simple) and [here](/developers/testing/scenario/values-complex).
+

docs/developers/testing/scenario/concept.md

@@ -0,0 +1,30 @@
+---
+id: generating-scenarios
+title: Generating scenarios
+---
+[comment]: # (mx-abstract)
+
+There are currently several ways to generate scenarios.
+
+The combination of generating and running scenarios is very powerful, since it means tests written originally for one system can be run of different systems too.
+
+This diagram shows all the currently possible paths.
+
+```mermaid
+graph TD
+    interact ----> blockchain["⚙️ Blockchain"]
+    interact[Interactor] -->|trace| json
+    manual[Manually implemented] --> json["JSON Scenario
+        <i>*.scen.json</i>"]
+    bb[BlackBox Test] -->|trace| json
+    wb[WhiteBox Test] -.->|trace| json
+    json --> test-go["Generated
+        <i>*_scenario_go_test.rs</i>"] --> vm-go["⚙️ Go VM"]
+    json --> test-rs["Generated
+        <i>*_scenario_rs_test.rs</i>"] --> vm-rust["⚙️ Rust VM (Debugger)"]
+    bb --> vm-rust
+    wb --> vm-rust
+```
+
+More details about this coming soon.
+