[E2E Testing] mage targets to interact with VMs and cloud stack

[belimawr]

The Goal

That's my OnWeek project, I focused on getting something simple that works, so I ended up not using most of the runner package/features because they're looked too coupled with running a test/set of tests and I was looking for easy and simple ways to interact directly with the VMs and Cloud Stacks.

The part installs tools on a VM is not very smart, it focus on not breaking if called multiple times rather than being super efficient.

What does this PR do?

This commit adds a number of mage targets to improve developer experience when using the integration test framework. Those targets aim to easy how to login into VMs and run tests from there as well as how to access the Cloud Stacks.

The targets added are:

• integration:deployDebugTools
• integration:deployEnvFile
• integration:generateEnvFile
• integration:listInstances
• integration:printState
• integration:ssh
• integration:stacks

Why is it important?

It improves developer experience

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
  - [ ] I have made corresponding change to the default configuration files
  - [ ] I have added tests that prove my fix is effective or that my feature works
  - [ ] I have added an entry in ./changelog/fragments using the changelog tool
  - [ ] I have added an integration test or an E2E test

## Author's Checklist

How to test this PR locally

Just run any of the new mage targets.

Related issues

• Blocked by: [8.12] Upgrades being incorrectly rolled back because the agent cannot parse the upgrade marker #3947
  ## Use cases
  ## Screenshots

Logs

Some example output from the new targets

mage integration:Stacks
+-------------------+------------------------------------------------------------------------------+ 
| #                 | 0                                                                            |
| Type              | stateful                                                                     |
| Deployment ID     | e73ae5d3fffff74aa436bbbbb8992342                                             |
| Elasticsearch URL | https://a-deployment.elastic-cloud.com                                       |
| Kibana            | https://a-deployment.elastic-cloud.com:9243                                  |
| Username          | elastic                                                                      |
| Password          | very-secret-password                                                         |
+-------------------+------------------------------------------------------------------------------+
+-------------------+--------------------------------------------------------+
| #                 | 1                                                      |
| Type              | serverless                                             |
| Project ID        | f2eddddd8d994aaa442677dc426cab235                      |
| Elasticsearch URL | https://a-serverless.es.elastic.cloud                  |
| Kibana            | https://a-serverless.kb.elastic.cloud                  |
| Username          | elastic                                                |
| Password          | very-secret-password                                   |
+-------------------+--------------------------------------------------------+

mage integration:ListInstances
+----------------+------------------------------------------------------------------------------------------------------------+
| #              | 0                                                                                                          |
| Provisioner    | ogc                                                                                                        |
| Name           | ogc-linux-amd64-ubuntu-2204-fleet-cac0                                                                     |
| ID             | linux-amd64-ubuntu-2204-fleet                                                                              |
| Instance ID    | 123458374283                                                                                               |
| IP             | 35.1.2.3                                                                                                   |
| Private Key    | /home/user/dev/elastic-agent/.integration-cache/id_rsa                                                     |
| Public Key     | /home/user/dev/elastic-agent/.integration-cache/id_rsa.pub                                                 |
| SSH connection | ssh -i /home/user/dev/elastic-agent/.integration-cache/id_rsa ubuntu@35.1.2.3                              |
| GCP Link       | https://console.cloud.google.com/compute/instancesDetail/zones/us-central1-a/instances/123458374283        |
+----------------+------------------------------------------------------------------------------------------------------------+
+----------------+-------------------------------------------------------------------------------------+
| #              | 1                                                                                   |
| Provisioner    | multipass                                                                           |
| Name           | linux-amd64-ubuntu-2204-fleet                                                       |
| ID             | linux-amd64-ubuntu-2204-fleet                                                       |
| IP             | 10.23.24.15                                                                         |
| Private Key    | /home/user/dev/elastic-agent/.integration-cache/id_rsa                              |
| Public Key     | /home/user/dev/elastic-agent/.integration-cache/id_rsa.pub                          |
| SSH connection | ssh -i /home/user/dev/elastic-agent/.integration-cache/id_rsa ubuntu@10.23.24.15    |
+----------------+-------------------------------------------------------------------------------------+

Questions to ask yourself

• How are we going to support this in production?
• How are we going to measure its adoption?
• How are we going to debug this?
• What are the metrics I should take care of?
• ...

[mergify]

This pull request does not have a backport label. Could you fix it @belimawr? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[blakerouse]

Overall these targets look very useful and could be very helpful for debugging. I gave some suggestions on the naming of the targets.

In general I think these items would be better inside of the pkg/testing/runner directly. Would be much cleaner then being into the mage file.

[belimawr]

Thanks for the suggestions @blakerouse! I implemented them on 8c06e69.

[belimawr]

In general I think these items would be better inside of the pkg/testing/runner directly. Would be much cleaner then being into the mage file.

I agree. I kept it in the magefile because it was a OnWeek project and I didn't have much time. I agree we should move it there later.

[pierrehilbert]

Setting back to draft for now.

[elastic-sonarqube]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube