Merge pull request #647 from galasa-dev/next

update main with changes for r 0.28.0 plus other doc enhancements

package.json

@@ -67,7 +67,7 @@
   "engines": {
     "node": "18.x"
   },
-  "overrides": { 
+  "overrides": {
     "lmdb": "2.6.0"
   }
 }

src/data/nav.yaml

@@ -36,6 +36,8 @@
               path: /docs/writing-own-tests/setting-up-galasa-project
             - title: Running a test locally
               path: /docs/cli-command-reference/cli-runs-submit-local
+            - title: Debugging a test locally
+              path: /docs/cli-command-reference/cli-runs-local-debug
         - title: Getting started using Eclipse
           path: /docs/getting-started/
           items:
@@ -67,6 +69,8 @@
           items:
             - title: Writing test classes
               path: /docs/writing-own-tests/writing-test-classes
+            - title: Running tests
+              path: /docs/writing-own-tests/running-test-modes
             - title: Testing across environments
               path: /docs/writing-own-tests/binding-tests
             - title: Test streams
@@ -86,6 +90,8 @@
               path: /docs/cli-command-reference/ecosystem-cli-runs-submit
             - title: Viewing test run status 
               path: /docs/cli-command-reference/cli-runs-get
+            - title: Downloading test artifacts 
+              path: /docs/cli-command-reference/ecosystem-cli-runs-download
         - title: Upgrading
           path: /docs/upgrading  
         - title: Managers

src/markdown-pages/docs/cli-command-reference/cli-command-reference.md

@@ -25,7 +25,7 @@ After downloading the binary, re-name it to `galasactl` (for Mac or Unix) or `ga
 
 Galasa CLI commands start with `galasactl`. Example commands are provided for running on Mac or Unix, and Windows Powershell. The Windows Powershell uses the backtick (`) for line continuation characters. If you are using Windows command-shell, the line continuation character is the caret (^). 
 
-You can view the Galasa CLI command syntax, including parameter descriptions, in the [Galasa cli repository](https://github.com/galasa-dev/cli/blob/main/docs/generated/galasactl.md).
+You can view the Galasa CLI command syntax, including parameter descriptions, in the <a href=https://github.com/galasa-dev/cli/blob/main/docs/generated/galasactl.md target="_blank"> cli repository</a> in GitHub.
 
 
 ## Known limitations
@@ -49,7 +49,9 @@ galasactl --log <logFilePath>
 
 ## Errors
 
-You can view a list of error messages that can be output by the galasactl tool in the [galasactl error documentation](https://github.com/galasa-dev/cli/blob/main/docs/generated/errors-list.md) in the cli repository in GitHub. 
+You can view a list of error messages that can be output by the galasactl tool in the <a href="https://github.com/galasa-dev/cli/blob/main/docs/generated/errors-list.md" target="_blank"> Galasa cli repository</a> in GitHub.
+
+
 
 
 

src/markdown-pages/docs/cli-command-reference/runs-download.md

@@ -0,0 +1,77 @@
+---
+path: "/docs/cli-command-reference/ecosystem-cli-runs-download"
+title: "Downloading test artifacts"
+---
+
+You can download the artifacts that are associated with a test run from the RAS and store those artifacts in a directory by using the `runs download` command. Use this output to gather information about a test, help debug and diagnose test failure, share test run output, and make comparisons between test run results.
+
+You can view the full list of options that are available with the `runs download` command in the 
+<a href="https://github.com/galasa-dev/cli/blob/main/docs/generated/galasactl_runs_download.md#galasactl-runs-download" target="_blank">Galasa cli repository</a>.
+
+## Working with the `runs download` command
+
+Downloaded artifacts are stored by default in a folder that is created within the current working directory from where the command is run. The name of the created folder corresponds to the test run-name that was provided. 
+
+In the following example, the test run is called _C1234_. All artifacts for test run `C1234` are downloaded to a folder named `C1234` in the current working directory by using the following command:
+
+```
+galasactl runs download --name C1234
+```
+
+If the folder named `C1234` does not exist, it is created. If artifacts are already stored within an existing folder named `C1234`, the download fails. You can force artifacts stored within an existing folder to be overwritten by using the `--force` flag:
+
+```
+galasactl runs download --name C1234 –-force
+```
+
+Use the `--force` flag carefully to avoid artifacts being over-written, resulting in the loss of data.
+
+### Specifying a folder for downloading artifacts
+
+You can specify a location where you want the test artifacts downloaded by using the `--destination` option on the `runs download` command. This is useful if you do not want to use the default directory. 
+
+The downloaded artifacts are organised within the specified directory in sub-folders that are based on the run-name. The file path of the folder can be relative to the current working directory, or absolute. For example, you can save artifacts from test run _C1234_ to a folder called _mytestruns_ in a directory called _temp_ by using the following command:
+
+```
+galasactl runs download --name C1234 --destination /temp/mytestruns
+```
+
+A message is displayed to confirm the location of the downloaded artifacts, for example:
+
+```
+GAL2501I: Downloaded 35 artifacts to '/temp/mytestruns/C1234' 
+```
+
+### Identifying output for incomplete test runs 
+
+If you are investigating a problem, you might want to download artifacts that are associated with a test run which has not finished. If a test run is not finished when the `runs download` command is submitted, artifact collection might not be complete.
+
+To identify such a scenario, a timestamp is appended to the folder name to indicate that the test did not finish running. The timestamp shows the time at which the `runs download` command is run. The time is displayed in Universal Time Coordinated (UTC) time. The date is displayed in the format _yyyy-mm-dd_. 
+
+For example, if the `runs download` command is run on the 6th of May 2023 at 10.30 am, against test _U5432_ which is still running, the created folder name would look similar to the following example:
+
+```
+U5432-2023-05-06_10:30:15
+```
+
+A message is displayed to show the location of any downloaded artifacts from the incomplete run, for example:
+
+```
+GAL2501I: Downloaded 11 files to folder U5432_2023-05-06_10:30:15
+```
+
+### Identifying output for test retries
+
+If a test is scheduled to run at a particular time but is unable to start, for example due to a lack of resources that are available in the environment, the ecosystem might retry the test at a later time. When investigating problems with a test running in an ecosystem, it is useful to download the artifacts that are associated with the retries of that test.
+
+When using the `runs download` command, if a test has run more than once, a number is added to the folder name to indicate the number of the retry, as shown in the following example:
+
+```
+C1234-1-2023-05-25_18:30:26
+C1234-2-2023-05-25_18:30:26
+C1234-3
+```
+
+In this example, the test _C1234_ tried to run twice unsuccessfully and completed on the third try. The numbers _1_ and _2_ in the folder names of the first two test run attempts indicate the retry order. The inclusion of the timestamp in folder name of the first two tries indicates that the test did not finish running. The third time the test finished running, so no timestamp is included as part of the name of the folder. 
+
+

src/markdown-pages/docs/cli-command-reference/runs-get.md

@@ -3,30 +3,142 @@ path: "/docs/cli-command-reference/cli-runs-get"
 title: "Viewing test run status"
 ---
 
+You can specify a number of options on the `galasactl runs get` command to query test run results on particular details, and to display the output of those queries in different formats. 
 
-Use the test run name to query test run details by using the following command:
+You can view the full command syntax in the <a href="https://github.com/galasa-dev/cli/blob/main/docs/generated/galasactl_runs_get.md" target="_blank">Galasa cli repository</a>.
+
+###  Options for querying and viewing test run results
+
+Use the tables provided to view the options for filtering test results, and for choosing the format in which you want those results displayed.
+
+<b>Table 1:</b> The following table shows the options that you can set on the `galasactl runs get` command to query test run data on specific details:
+
+| Name |  Description  |
+| :---- | :-------- | 
+| `--name`  | Use the `--name` option to query the status of a particular test run.  |
+| `--age`| Use the `--age` option to specify a time period in which the tests ran. The _age_ option is specified in the format _FROM:TO_. Units of time can be specified in weeks _w_, days _d_, or hours _h_. The _FROM_ part is mandatory. The _TO_ part is optional, with a default set to `0`, which indicates the current time. The _FROM_ value specifies how far back in time the query is applied. The _FROM_ value must therefore always be a larger value than the _TO_ value. If the `--name` option is specified, the `--age` parameter is not used. |
+
+
+<b>Table 2:</b> The following table shows the options that you can set on the `galasactl runs get` command to display test run results in different formats:
+
+| Name |  Description  |
+| :---- | :-------- | 
+| `--format summary` | The default format is _summary_. The _summary_ format is useful if you just need to know the high-level status and result of a test.  | 
+| `--format details` | The _details_ format is useful when you need to see all the details of a test run, including method information.  | 
+| `--format raw` | The _raw_ format output is useful if you are writing scripts to report on multiple test runs programmatically. The output from `galasactl runs get` is returned in a form that makes it easy for scripting to digest the individual pieces of data available. | 
+
+
+## Examples
+
+Use the following sections to view some example command options and generated output. Note that the `--format summary` and `--format details` options return a total count of results returned, along with a breakdown of the number that are returned with a particular result, for example, _Passed_, _Failed_, _PassedWithDefects_. 
+
+### View tests results in summary format
+
+This is the default format. Use this format to get a quick, high-level update on the status of a test run or runs. Returned information includes details about time the test was submitted, run name, status, result, and test name. This format is especially useful if a query returns a large number of results. 
+
+The following example command returns test status in a summary format for tests that ran between two weeks ago and one week ago:
 
 ```
-galasactl runs get --runname <NameOfTestRun> --bootstrap <BootstrapURL>
+galasactl runs get --bootstrap http://example.com:30960/boostrap --age 2w:1w
 ``` 
 
 where:
-- `--runname` is the name of the test run 
+
 - `--bootstrap` is the URL of the ecosystem's bootstrap properties
+- `--age` is the period of time in which the tests ran
 
 
 Results are returned on the terminal in the following example format:
 
 ```
-galasactl runs get --runname U456 
-RunName Status   Result ShortTestName
-U456    Finished Passed MyTestName
+$galasactl runs get --bootstrap http://example.com:30960/boostrap --age 2w:1w --format summary
+
+submitted-time      name status   result test-name
+2023-05-04 10:55:29 U456 Finished Passed MyTestName1
+2023-05-05 10:45:29 U856 Finished Passed MyTestName2
+2023-05-06 11:55:29 U859 Finished Passed MyTestName3
+2023-05-07 10:55:23 U956 Finished Passed MyTestName4
+2023-05-07 10:56:29 U976 Finished Passed MyTestName5
+2023-05-07 10:57:20 U996 Finished Passed MyTestName6
+
+Total:6 Passed:6 
 ```
 
+### View tests results in details format
+
+Use this format to drill down to get more details on a particular test run. The returned information includes details about the bundle and a link to the run log URL, which can be viewed in a browser. Viewing the run log in this way makes it easier to diagnose why a test failed. A table of methods is also displayed, including details about the status and result of each test method. 
+
+The following example command returns test status in a details format:
+
+On Mac or Unix:
 
+```
+$galasactl runs get --name U456 --bootstrap http://example.com:30960/boostrap \
+--format details 
+``` 
 
+On Windows (Powershell):
+```
+galasactl runs get --name U456 --bootstrap http://example.com:30960/boostrap `
+--format details 
+``` 
 
+where:
+- `--name` is the name of the test run 
+- `--bootstrap` is the URL of the ecosystem's bootstrap properties
+- `--format` is the format in which you want the results to display
 
 
+Results are returned on the terminal in the following example format:
+
+```
+$galasactl runs get --name U456 --format details
+name           : U456
+status         : Finished
+result         : Passed
+submitted-time : 2023-05-04 10:55:29
+start-time     : 2023-05-05 06:00:14
+end-time       : 2023-05-05 06:00:15
+duration(ms)   : 1000
+test-name      : dev.galasa.Zos3270LocalJava11Ubuntu
+bundle         : dev.galasa
+run-log        : https://127.0.0.1/ras/run/cbd-123/runlog
+
+method          type status   result start-time          end-time            duration(ms)
+testCoreIvtTest test Finished Passed 2023-05-05 06:03:38 2023-05-05 06:03:39 1000
+
+Total:1 Passed:1  
+```
+
+### View tests results in raw format
+
+Use this format if you want to parse test results using scripts. The raw format returns values separated by pipes, without formatting or header information.
+
+The following example command returns test status in a raw format:
+
+On Mac or Unix:
+
+```
+galasactl runs get --name U456 --bootstrap http://example.com:30960/boostrap \
+--format raw
+``` 
+
+On Windows (Powershell):
+```
+galasactl runs get --name U456 --bootstrap http://example.com:30960/boostrap \
+--format raw
+``` 
+
+where:
+- `--name` is the name of the test run 
+- `--bootstrap` is the URL of the ecosystem's bootstrap properties
+- `--format` is the format in which you want the results to display
+
+Results are returned on the terminal in the following example format:
+
+```
+$galasactl runs get --name U456 --format raw
+U456|Finished|Passed|2023-05-04T10:55:29.545323Z|2023-05-05T06:00:14.496953Z|2023-05-05T06:00:15.654565Z|1157|dev.galasa.Zos3270LocalJava11Ubuntu|galasa|dev.galasa|https://127.0.0.1/ras/run/cbd-123/runlog
+```