diff --git a/test/e2e/README.md b/test/e2e/README.md index 80c647bd79a..eb9d4a9857f 100644 --- a/test/e2e/README.md +++ b/test/e2e/README.md @@ -1,8 +1,8 @@ # E2E testing | Flavor | Results | |--|--| -|**Full mode** | | -|**Docker compose** | +|**Full mode** | | +|**Docker compose** | E2E functional tests of cardano-wallet are running nightly on [cardano preprod testnet](https://book.world.dev.cardano.org/environments.html). Running tests against public testnet allows to exercise cardano-wallet on environment close to production (mainnet) utilizing and integrating maximally all components of the Cardano ecosystem like Stake pools, SMASH, metadata token server etc. @@ -24,11 +24,11 @@ bundle install export TESTS_E2E_FIXTURES=******* rake secrets_decode ``` -> :information_source: **_TESTS_E2E_FIXTURES_** secret is defined on https://github.com/input-output-hk/cardano-wallet/settings/secrets and also used by GH actions. Note that this step is also executed on very first test run. +> :information_source: **_TESTS_E2E_FIXTURES_** secret is defined on https://github.com/cardano-foundation/cardano-wallet/settings/secrets and also used by GH actions. Note that this step is also executed on very first test run. > :information_source: **_TESTS_E2E_FIXTURES_** is also kept in the team's Vaultwarden. #### Fixture wallets -The command `rake secrets_decode` can be used to decode the file [fixture_wallets.json.gpg](https://github.com/input-output-hk/cardano-wallet/blob/master/test/e2e/fixtures/fixture_wallets.json.gpg) into a plain JSON file named `fixture_wallets.json`. This file contains mnemonic sentences of the wallets that are utilized in tests. The test framework will retrieve these wallets from the file. It is important to note that there are separate sets of wallets for different operating systems, namely Linux, MacOS, and Windows. This allows tests to be run in parallel on different operating systems. +The command `rake secrets_decode` can be used to decode the file [fixture_wallets.json.gpg](https://github.com/cardano-foundation/cardano-wallet/blob/master/test/e2e/fixtures/fixture_wallets.json.gpg) into a plain JSON file named `fixture_wallets.json`. This file contains mnemonic sentences of the wallets that are utilized in tests. The test framework will retrieve these wallets from the file. It is important to note that there are separate sets of wallets for different operating systems, namely Linux, MacOS, and Windows. This allows tests to be run in parallel on different operating systems. #### Fixture wallets update If there is a need to update the mnemonics to new ones, it is certainly possible. After making the updates, remember to use the `rake secrets_encode` command to encode the updated plain JSON file into a GPG-encoded file. The updated GPG file then needs to be pushed to the repository so that it can be used by the continuous integration (CI) system. @@ -40,7 +40,7 @@ Please note that the wallets used in tests must have ADA and specific assets on - SadCoin (with [metadata](https://metadata.cardano-testnet.iohkdev.io/metadata/ee1ce9d7560f48a4ba3867037dbec2d8fed776d94dd6b00a35309073)) - HappyCoin (with [metadata](https://metadata.cardano-testnet.iohkdev.io/metadata/919e8a1922aaa764b1d66407c6f62244e77081215f385b60a62091494861707079436f696e)) -Both assets have metadata in the [Testnet Metadata Registry](https://github.com/input-output-hk/metadata-registry-testnet), and there are tests in place to ensure that the wallet reads data from there correctly. Both assets can be minted outside of the wallet using tools such as `cardano-cli` or [token-minter](https://github.com/piotr-iohk/token_minter), and then sent to the balances of the fixture wallets. Policy scripts and keys required for minting are availeble in [tests/e2e/fixtuers/wallet_assets](https://github.com/input-output-hk/cardano-wallet/tree/master/test/e2e/fixtures/wallet_assets). +Both assets have metadata in the [Testnet Metadata Registry](https://github.com/input-output-hk/metadata-registry-testnet), and there are tests in place to ensure that the wallet reads data from there correctly. Both assets can be minted outside of the wallet using tools such as `cardano-cli` or [token-minter](https://github.com/piotr-iohk/token_minter), and then sent to the balances of the fixture wallets. Policy scripts and keys required for minting are availeble in [tests/e2e/fixtuers/wallet_assets](https://github.com/cardano-foundation/cardano-wallet/tree/master/test/e2e/fixtures/wallet_assets). #### Further test setup There may be certain tests that require additional actions to be performed before they can be successfully executed. In such cases, the test will fail and display an appropriate message with instructions. An example of such a test is `AlwaysFails.plutus with collateral return to the wallet`, which requires a specific UTXO to be set up on the testnet where the test will be executed. This setup is a one-time operation and is only necessary when running tests on a new testnet or when the existing testnet has been reset. The scenario is accompanied by detailed comments and step-by-step instructions on how to proceed. If the required actions have not been taken, the test will fail and display the appropriate message. @@ -77,7 +77,7 @@ One can also start tests against cardano-wallet docker. There is docker-compose- >NETWORK=preprod \ >TESTS_E2E_TOKEN_METADATA=https://metadata.cardano-testnet.iohkdev.io/ \ >WALLET=dev-master \ ->NODE=1.30.1 \ +>NODE=8.1.1 \ >NODE_CONFIG_PATH=`pwd`/state/configs/$NETWORK \ >DATA=`pwd`/state/node_db/$NETWORK >docker-compose -f docker-compose-test.yml up @@ -94,12 +94,12 @@ One can also run tests against `cardano-wallet` and `cardano-node` which are spe ```bash $ TESTS_E2E_BINDIR="" rake run_on[testnet] ``` -Running tests as such skips downloading latest binaries from Hydra. +Running tests as such skips downloading latest wallet and node binaries. ### Test artifacts By default following locations are used for different artifacts used by the tests: -- `./bins` - location for wallet and node binaries (will be downloaded here from [Hydra](https://hydra.iohk.io/jobset/Cardano/cardano-wallet#tabs-jobs)) +- `./bins` - location for wallet and node binaries (will be downloaded here from [Buildkite](https://buildkite.com/cardanofoundation/cardano-wallet)) - `./state` - wallet/node databases, logs and configs Locations are relative to `test/e2e` directory. @@ -175,7 +175,7 @@ For speeding up execution in wallet's full mode we use cardano-node DB from cach All tests are scheduled to be executed on nightly basis against latest `master` version of cardano-wallet. -It is also possible to trigger each workflow manually from [GH actions](https://github.com/input-output-hk/cardano-wallet/actions). In particular: +It is also possible to trigger each workflow manually from [GH actions](https://github.com/cardano-foundation/cardano-wallet/actions). In particular: - workflows can be executed against the binaries of particular PR, - for full wallet mode one can choose whether to use Node DB cached from GH cache or AWS diff --git a/test/e2e/Rakefile b/test/e2e/Rakefile index 47d6bde286d..0729e135e52 100644 --- a/test/e2e/Rakefile +++ b/test/e2e/Rakefile @@ -176,11 +176,30 @@ task :start_node_and_wallet, [:env] do |_task, args| cmd start_node cmd start_wallet else - start_node = "#{bin_dir}cardano-node run --config #{config_dir}/config.json --topology #{config_dir}/topology.json --database-path #{node_db_dir} --socket-path #{node_db_dir}/node.socket" - start_wallet = "#{bin_dir}cardano-wallet serve --port #{WALLET_PORT} --node-socket #{node_db_dir}/node.socket #{network} --database #{wallet_db_dir} --token-metadata-server #{TOKEN_METADATA}" - - cmd "screen -dmS NODE_#{args[:env]} -L -Logfile #{log_dir}/node.log #{start_node}" - cmd "screen -dmS WALLET_#{args[:env]} -L -Logfile #{log_dir}/wallet.log #{start_wallet}" + start_node = <<~CMD + #{bin_dir}cardano-node run \ + --config #{config_dir}/config.json \ + --topology #{config_dir}/topology.json \ + --database-path #{node_db_dir} \ + --socket-path #{node_db_dir}/node.socket + CMD + start_wallet = <<~CMD + #{bin_dir}cardano-wallet serve #{network} \ + --port #{WALLET_PORT} \ + --node-socket #{node_db_dir}/node.socket \ + --database #{wallet_db_dir} \ + --token-metadata-server #{TOKEN_METADATA} + CMD + cmd <<~CMD + screen -dmS NODE_#{args[:env]} -L \ + -Logfile #{log_dir}/node.log \ + #{start_node} + CMD + cmd <<~CMD + screen -dmS WALLET_#{args[:env]} -L \ + -Logfile #{log_dir}/wallet.log \ + #{start_wallet} + CMD cmd 'screen -ls', display_result: true end end @@ -296,19 +315,13 @@ task :get_latest_configs, [:env] do |_task, args| wget("#{base_url}byron-genesis.json", "#{out}/byron-genesis.json") wget("#{base_url}shelley-genesis.json", "#{out}/shelley-genesis.json") wget("#{base_url}alonzo-genesis.json", "#{out}/alonzo-genesis.json") + wget("#{base_url}conway-genesis.json", "#{out}/conway-genesis.json") wget("#{base_url}topology.json", "#{out}/topology.json") - # remove environment names from genesis files - config = File.read(config_file) - config_edited = config.gsub(/#{env}-/, '') - File.open(config_file, 'w') { |file| file.puts config_edited } - # Temporary/ad-hoc config changes for cardano-node: - # - disable P2P - # - use conway-genesis.json config_dir = File.join(CONFIGS, args[:env]) + # Temporary/ad-hoc config changes for cardano-node: config_json = JSON.parse(File.read("#{config_dir}/config.json")) config_json['EnableP2P'] = false - config_json['ConwayGenesisFile'] = 'conway-genesis.json' File.write("#{config_dir}/config.json", JSON.pretty_generate(config_json)) topology = %({ @@ -321,10 +334,6 @@ task :get_latest_configs, [:env] do |_task, args| ] }) File.write("#{config_dir}/topology.json", topology) - - # Add conway-genesis.json - File.write("#{config_dir}/conway-genesis.json", ' { "genDelegs": {} }') - end end diff --git a/test/e2e/helpers/buildkite.rb b/test/e2e/helpers/buildkite.rb index 7cb90b2382f..18ea0d14de2 100644 --- a/test/e2e/helpers/buildkite.rb +++ b/test/e2e/helpers/buildkite.rb @@ -12,7 +12,7 @@ class Buildkite attr_reader :org, :pipeline, :client def initialize(pipeline = 'cardano-wallet') - @org = 'input-output-hk' + @org = 'cardanofoundation' @pipeline = pipeline @api_token = ENV.fetch('BUILDKITE_API_TOKEN', nil) @client = Buildkit.new(token: @api_token) diff --git a/test/e2e/spec/shelley_spec.rb b/test/e2e/spec/shelley_spec.rb index d7698ec192c..db329520a16 100644 --- a/test/e2e/spec/shelley_spec.rb +++ b/test/e2e/spec/shelley_spec.rb @@ -414,16 +414,14 @@ expect(s).to be_correct_and_respond 204 eventually "Pools have metadata when 'pool_metadata_source' => 'direct'" do - sps = pools.list({ stake: 1000 }) - !sps.select { |p| p['metadata'] }.empty? + pools.list({ stake: 1000 }).any? { |p| p['metadata'] } end s = settings.update({ pool_metadata_source: 'none' }) expect(s).to be_correct_and_respond 204 eventually "Pools have no metadata when 'pool_metadata_source' => 'none'" do - sps = pools.list({ stake: 1000 }) - sps.select { |p| p['metadata'] }.empty? + pools.list({ stake: 1000 }).none? { |p| p['metadata'] } end end @@ -435,16 +433,14 @@ expect(s).to be_correct_and_respond 204 eventually "Pools have metadata when 'pool_metadata_source' => '#{ENV.fetch('TESTS_E2E_SMASH', nil)}'" do - sps = pools.list({ stake: 1000 }) - !sps.select { |p| p['metadata'] }.empty? + pools.list({ stake: 1000 }).any? { |p| p['metadata'] } end s = settings.update({ pool_metadata_source: 'none' }) expect(s).to be_correct_and_respond 204 eventually "Pools have no metadata when 'pool_metadata_source' => 'none'" do - sps = pools.list({ stake: 1000 }) - sps.select { |p| p['metadata'] }.empty? + pools.list({ stake: 1000 }).none? { |p| p['metadata'] } end end end diff --git a/test/e2e/spec/spec_helper.rb b/test/e2e/spec/spec_helper.rb index 24874894400..45734434924 100644 --- a/test/e2e/spec/spec_helper.rb +++ b/test/e2e/spec/spec_helper.rb @@ -7,6 +7,7 @@ require 'mustache' require 'cbor' require 'tmpdir' +require 'set' require_relative '../env' require_relative '../helpers/utils' require_relative '../helpers/tx_history'