Merge pull request #4012 from cardano-foundation/yura/fix-e2e-tests

E2E fix to reflect that buildkite moved from input-output-hk to cardanofoundation org

test/e2e/README.md

@@ -1,8 +1,8 @@
 # E2E testing
 | Flavor | Results |
 |--|--|
-|**Full mode** |<a href="https://github.com/input-output-hk/cardano-wallet/actions/workflows/e2e-docker.yml"><img src="https://img.shields.io/github/actions/workflow/status/input-output-hk/cardano-wallet/e2e-docker.yml?label=E2E Docker&style=for-the-badge&branch=master"  /></a> <a href="https://github.com/input-output-hk/cardano-wallet/actions/workflows/e2e-linux.yml"><img src="https://img.shields.io/github/actions/workflow/status/input-output-hk/cardano-wallet/e2e-linux.yml?label=E2E Linux&style=for-the-badge&branch=master"  /></a> <a href="https://github.com/input-output-hk/cardano-wallet/actions/workflows/e2e-macos.yml"><img src="https://img.shields.io/github/actions/workflow/status/input-output-hk/cardano-wallet/e2e-macos.yml?label=E2E MacOs&style=for-the-badge&branch=master"  /></a> <a href="https://github.com/input-output-hk/cardano-wallet/actions/workflows/e2e-windows.yml"><img src="https://img.shields.io/github/actions/workflow/status/input-output-hk/cardano-wallet/e2e-windows.yml?label=E2E Windows&style=for-the-badge&branch=master" /></a> |
-|**Docker compose** | <a href="https://github.com/input-output-hk/cardano-wallet/actions/workflows/docker_linux.yml"><img src="https://img.shields.io/github/actions/workflow/status/input-output-hk/cardano-wallet/docker_linux.yml?label=Docker-compose Linux&style=for-the-badge&branch=master"  /></a>  <a href="https://github.com/input-output-hk/cardano-wallet/actions/workflows/docker_macos.yml"><img src="https://img.shields.io/github/actions/workflow/status/input-output-hk/cardano-wallet/docker_macos.yml?label=Docker-compose Macos&style=for-the-badge&branch=master"  /></a>
+|**Full mode** |<a href="https://github.com/cardano-foundation/cardano-wallet/actions/workflows/e2e-docker.yml"><img src="https://img.shields.io/github/actions/workflow/status/cardano-foundation/cardano-wallet/e2e-docker.yml?label=E2E Docker&style=for-the-badge&branch=master"  /></a> <a href="https://github.com/cardano-foundation/cardano-wallet/actions/workflows/e2e-linux.yml"><img src="https://img.shields.io/github/actions/workflow/status/cardano-foundation/cardano-wallet/e2e-linux.yml?label=E2E Linux&style=for-the-badge&branch=master"  /></a> <a href="https://github.com/cardano-foundation/cardano-wallet/actions/workflows/e2e-macos.yml"><img src="https://img.shields.io/github/actions/workflow/status/cardano-foundation/cardano-wallet/e2e-macos.yml?label=E2E MacOs&style=for-the-badge&branch=master"  /></a> <a href="https://github.com/cardano-foundation/cardano-wallet/actions/workflows/e2e-windows.yml"><img src="https://img.shields.io/github/actions/workflow/status/cardano-foundation/cardano-wallet/e2e-windows.yml?label=E2E Windows&style=for-the-badge&branch=master" /></a> |
+|**Docker compose** | <a href="https://github.com/cardano-foundation/cardano-wallet/actions/workflows/docker_linux.yml"><img src="https://img.shields.io/github/actions/workflow/status/cardano-foundation/cardano-wallet/docker_linux.yml?label=Docker-compose Linux&style=for-the-badge&branch=master"  /></a>  <a href="https://github.com/cardano-foundation/cardano-wallet/actions/workflows/docker_macos.yml"><img src="https://img.shields.io/github/actions/workflow/status/cardano-foundation/cardano-wallet/docker_macos.yml?label=Docker-compose Macos&style=for-the-badge&branch=master"  /></a>
 
 E2E functional tests of cardano-wallet are running nightly on [cardano preprod testnet](https://book.world.dev.cardano.org/environments.html). Running tests against public testnet allows to exercise cardano-wallet on environment close to production (mainnet) utilizing and integrating maximally all components of the Cardano ecosystem like Stake pools, SMASH, metadata token server etc.
 
@@ -24,11 +24,11 @@ bundle install
 export TESTS_E2E_FIXTURES=*******
 rake secrets_decode
 ```
-> :information_source:  **_TESTS_E2E_FIXTURES_** secret  is defined on https://github.com/input-output-hk/cardano-wallet/settings/secrets and also used by GH actions. Note that this step is also executed on very first test run.
+> :information_source:  **_TESTS_E2E_FIXTURES_** secret  is defined on https://github.com/cardano-foundation/cardano-wallet/settings/secrets and also used by GH actions. Note that this step is also executed on very first test run.
 > :information_source:  **_TESTS_E2E_FIXTURES_** is also kept in the team's Vaultwarden.
 
 #### Fixture wallets
-The command `rake secrets_decode` can be used to decode the file [fixture_wallets.json.gpg](https://github.com/input-output-hk/cardano-wallet/blob/master/test/e2e/fixtures/fixture_wallets.json.gpg) into a plain JSON file named `fixture_wallets.json`. This file contains mnemonic sentences of the wallets that are utilized in tests. The test framework will retrieve these wallets from the file. It is important to note that there are separate sets of wallets for different operating systems, namely Linux, MacOS, and Windows. This allows tests to be run in parallel on different operating systems.
+The command `rake secrets_decode` can be used to decode the file [fixture_wallets.json.gpg](https://github.com/cardano-foundation/cardano-wallet/blob/master/test/e2e/fixtures/fixture_wallets.json.gpg) into a plain JSON file named `fixture_wallets.json`. This file contains mnemonic sentences of the wallets that are utilized in tests. The test framework will retrieve these wallets from the file. It is important to note that there are separate sets of wallets for different operating systems, namely Linux, MacOS, and Windows. This allows tests to be run in parallel on different operating systems.
 
 #### Fixture wallets update
 If there is a need to update the mnemonics to new ones, it is certainly possible. After making the updates, remember to use the `rake secrets_encode` command to encode the updated plain JSON file into a GPG-encoded file. The updated GPG file then needs to be pushed to the repository so that it can be used by the continuous integration (CI) system.
@@ -40,7 +40,7 @@ Please note that the wallets used in tests must have ADA and specific assets on
 -   SadCoin (with [metadata](https://metadata.cardano-testnet.iohkdev.io/metadata/ee1ce9d7560f48a4ba3867037dbec2d8fed776d94dd6b00a35309073))
 -   HappyCoin (with [metadata](https://metadata.cardano-testnet.iohkdev.io/metadata/919e8a1922aaa764b1d66407c6f62244e77081215f385b60a62091494861707079436f696e))
 
-Both assets have metadata in the [Testnet Metadata Registry](https://github.com/input-output-hk/metadata-registry-testnet), and there are tests in place to ensure that the wallet reads data from there correctly. Both assets can be minted outside of the wallet using tools such as `cardano-cli` or [token-minter](https://github.com/piotr-iohk/token_minter), and then sent to the balances of the fixture wallets. Policy scripts and keys required for minting are availeble in [tests/e2e/fixtuers/wallet_assets](https://github.com/input-output-hk/cardano-wallet/tree/master/test/e2e/fixtures/wallet_assets).
+Both assets have metadata in the [Testnet Metadata Registry](https://github.com/input-output-hk/metadata-registry-testnet), and there are tests in place to ensure that the wallet reads data from there correctly. Both assets can be minted outside of the wallet using tools such as `cardano-cli` or [token-minter](https://github.com/piotr-iohk/token_minter), and then sent to the balances of the fixture wallets. Policy scripts and keys required for minting are availeble in [tests/e2e/fixtuers/wallet_assets](https://github.com/cardano-foundation/cardano-wallet/tree/master/test/e2e/fixtures/wallet_assets).
 
 #### Further test setup 
 There may be certain tests that require additional actions to be performed before they can be successfully executed. In such cases, the test will fail and display an appropriate message with instructions. An example of such a test is `AlwaysFails.plutus with collateral return to the wallet`, which requires a specific UTXO to be set up on the testnet where the test will be executed. This setup is a one-time operation and is only necessary when running tests on a new testnet or when the existing testnet has been reset. The scenario is accompanied by detailed comments and step-by-step instructions on how to proceed. If the required actions have not been taken, the test will fail and display the appropriate message.
@@ -77,7 +77,7 @@ One can also start tests against cardano-wallet docker. There is docker-compose-
 >NETWORK=preprod \
 >TESTS_E2E_TOKEN_METADATA=https://metadata.cardano-testnet.iohkdev.io/ \
 >WALLET=dev-master \
->NODE=1.30.1 \
+>NODE=8.1.1 \
 >NODE_CONFIG_PATH=`pwd`/state/configs/$NETWORK \
 >DATA=`pwd`/state/node_db/$NETWORK
 >docker-compose -f docker-compose-test.yml up
@@ -94,12 +94,12 @@ One can also run tests against `cardano-wallet` and `cardano-node` which are spe
 ```bash
 $ TESTS_E2E_BINDIR="" rake run_on[testnet]
 ```
-Running tests as such skips downloading latest binaries from Hydra.
+Running tests as such skips downloading latest wallet and node binaries.
 
 ### Test artifacts
 
 By default following locations are used for different artifacts used by the tests:
-- `./bins` - location for wallet and node binaries (will be downloaded here from [Hydra](https://hydra.iohk.io/jobset/Cardano/cardano-wallet#tabs-jobs))
+- `./bins` - location for wallet and node binaries (will be downloaded here from [Buildkite](https://buildkite.com/cardanofoundation/cardano-wallet))
 - `./state` - wallet/node databases, logs and configs
 
 Locations are relative to `test/e2e` directory.
@@ -175,7 +175,7 @@ For speeding up execution in wallet's full mode we use cardano-node DB from cach
 All tests are scheduled to be executed on nightly basis against latest `master` version
 of cardano-wallet.
 
-It is also possible to trigger each workflow manually from [GH actions](https://github.com/input-output-hk/cardano-wallet/actions). In particular:
+It is also possible to trigger each workflow manually from [GH actions](https://github.com/cardano-foundation/cardano-wallet/actions). In particular:
  - workflows can be executed against the binaries of particular PR,
  - for full wallet mode one can choose whether to use Node DB cached from GH cache or AWS
 

test/e2e/Rakefile

@@ -176,11 +176,30 @@ task :start_node_and_wallet, [:env] do |_task, args|
     cmd start_node
     cmd start_wallet
   else
-    start_node = "#{bin_dir}cardano-node run --config #{config_dir}/config.json --topology #{config_dir}/topology.json --database-path #{node_db_dir} --socket-path #{node_db_dir}/node.socket"
-    start_wallet = "#{bin_dir}cardano-wallet serve --port #{WALLET_PORT} --node-socket #{node_db_dir}/node.socket #{network} --database #{wallet_db_dir} --token-metadata-server #{TOKEN_METADATA}"
-
-    cmd "screen -dmS NODE_#{args[:env]} -L -Logfile #{log_dir}/node.log #{start_node}"
-    cmd "screen -dmS WALLET_#{args[:env]} -L -Logfile #{log_dir}/wallet.log #{start_wallet}"
+    start_node = <<~CMD
+      #{bin_dir}cardano-node run \
+        --config #{config_dir}/config.json \
+        --topology #{config_dir}/topology.json \
+        --database-path #{node_db_dir} \
+        --socket-path #{node_db_dir}/node.socket
+    CMD
+    start_wallet = <<~CMD
+      #{bin_dir}cardano-wallet serve #{network} \
+        --port #{WALLET_PORT} \
+        --node-socket #{node_db_dir}/node.socket \
+        --database #{wallet_db_dir} \
+        --token-metadata-server #{TOKEN_METADATA}
+    CMD
+    cmd <<~CMD
+      screen -dmS NODE_#{args[:env]} -L \
+        -Logfile #{log_dir}/node.log \
+        #{start_node}
+    CMD
+    cmd <<~CMD
+      screen -dmS WALLET_#{args[:env]} -L \
+        -Logfile #{log_dir}/wallet.log \
+        #{start_wallet}
+    CMD
     cmd 'screen -ls', display_result: true
   end
 end
@@ -296,19 +315,13 @@ task :get_latest_configs, [:env] do |_task, args|
     wget("#{base_url}byron-genesis.json", "#{out}/byron-genesis.json")
     wget("#{base_url}shelley-genesis.json", "#{out}/shelley-genesis.json")
     wget("#{base_url}alonzo-genesis.json", "#{out}/alonzo-genesis.json")
+    wget("#{base_url}conway-genesis.json", "#{out}/conway-genesis.json")
     wget("#{base_url}topology.json", "#{out}/topology.json")
-    # remove environment names from genesis files
-    config = File.read(config_file)
-    config_edited = config.gsub(/#{env}-/, '')
-    File.open(config_file, 'w') { |file| file.puts config_edited }
 
-    # Temporary/ad-hoc config changes for cardano-node:
-    #  - disable P2P
-    #  - use conway-genesis.json
     config_dir = File.join(CONFIGS, args[:env])
+    # Temporary/ad-hoc config changes for cardano-node:
     config_json = JSON.parse(File.read("#{config_dir}/config.json"))
     config_json['EnableP2P'] = false
-    config_json['ConwayGenesisFile'] = 'conway-genesis.json'
     File.write("#{config_dir}/config.json", JSON.pretty_generate(config_json))
 
     topology = %({
@@ -321,10 +334,6 @@ task :get_latest_configs, [:env] do |_task, args|
           ]
         })
     File.write("#{config_dir}/topology.json", topology)
-
-    # Add conway-genesis.json
-    File.write("#{config_dir}/conway-genesis.json", ' { "genDelegs": {} }')
-
   end
 end
 

test/e2e/helpers/buildkite.rb

@@ -12,7 +12,7 @@ class Buildkite
   attr_reader :org, :pipeline, :client
 
   def initialize(pipeline = 'cardano-wallet')
-    @org = 'input-output-hk'
+    @org = 'cardanofoundation'
     @pipeline = pipeline
     @api_token = ENV.fetch('BUILDKITE_API_TOKEN', nil)
     @client = Buildkit.new(token: @api_token)

test/e2e/spec/shelley_spec.rb

@@ -414,16 +414,14 @@
         expect(s).to be_correct_and_respond 204
 
         eventually "Pools have metadata when 'pool_metadata_source' => 'direct'" do
-          sps = pools.list({ stake: 1000 })
-          !sps.select { |p| p['metadata'] }.empty?
+          pools.list({ stake: 1000 }).any? { |p| p['metadata'] }
         end
 
         s = settings.update({ pool_metadata_source: 'none' })
         expect(s).to be_correct_and_respond 204
 
         eventually "Pools have no metadata when 'pool_metadata_source' => 'none'" do
-          sps = pools.list({ stake: 1000 })
-          sps.select { |p| p['metadata'] }.empty?
+          pools.list({ stake: 1000 }).none? { |p| p['metadata'] }
         end
       end
 
@@ -435,16 +433,14 @@
         expect(s).to be_correct_and_respond 204
 
         eventually "Pools have metadata when 'pool_metadata_source' => '#{ENV.fetch('TESTS_E2E_SMASH', nil)}'" do
-          sps = pools.list({ stake: 1000 })
-          !sps.select { |p| p['metadata'] }.empty?
+          pools.list({ stake: 1000 }).any? { |p| p['metadata'] }
         end
 
         s = settings.update({ pool_metadata_source: 'none' })
         expect(s).to be_correct_and_respond 204
 
         eventually "Pools have no metadata when 'pool_metadata_source' => 'none'" do
-          sps = pools.list({ stake: 1000 })
-          sps.select { |p| p['metadata'] }.empty?
+          pools.list({ stake: 1000 }).none? { |p| p['metadata'] }
         end
       end
     end

test/e2e/spec/spec_helper.rb

@@ -7,6 +7,7 @@
 require 'mustache'
 require 'cbor'
 require 'tmpdir'
+require 'set'
 require_relative '../env'
 require_relative '../helpers/utils'
 require_relative '../helpers/tx_history'