added value provider scripts

README.md

@@ -28,14 +28,21 @@ Note:  Digital.ai version should not be lower than lowest supported version. Thi
 
 ## Features/Usage/Types/Tasks
 
-This plugin adds a new configuration type, Mabl Server, to the list of available HTTP Connection Server Types.
+This plugin adds a new configuration type, Mabl Server Workspace, to the list of available HTTP Connection Server Types. You must create a Mabl Server Workspace for each workspace you wish to interact with even if they reside on the same Mabl server.
 
 ![mabl_serverType](images/serverType.png)
 
-Before configuring the Mabl Server, create an API Token, of type 'Deployment Trigger Api Key' from [https://app.mabl.com](https://app.mabl.com). Configure the Server URL field with the value used for API calls, namely [https://api.mabl.com](https://api.mabl.com). The 'Test Connection' test associated with the Mabl Server simply makes a call to the url [https://app.mabl.com/login](https://app.mabl.com/login) to ensure Release can reach the Mabl site. Note: The connection test does not test the validity of the API Token.
+Before configuring the Mabl Server Workspace, create two API Tokens, one of type 'Deployment Trigger Api Key' from [https://app.mabl.com](https://app.mabl.com) and one of type CI/CD Integration. Configure the Server URL field with the value used for API calls, namely [https://api.mabl.com](https://api.mabl.com). The screenshot below shows the page within the Mabl App where tokens can be created.
 
-![mabl_serverConfiguration](images/serverConfiguration.png)
+![mabl_MablTokenConfig](images/MablTokenConfig.png)
+
+You will also need to know the workspace id. The screenshot below shows where to find the workspace id within the Mabl App.
+
+![mabl_mablWorkspaceId](images/mablWorkspaceId.png)
 
+Note: The 'Test Connection' test associated with the Mabl Server Workspace simply makes a call to the url [https://app.mabl.com/login](https://app.mabl.com/login) to ensure Release can reach the Mabl site. Note: The connection test does not test the validity of the API Tokens or the Workspace Id.
+
+![mabl_serverConfiguration](images/serverConfiguration.png)
 
 The Mabl plugin adds the following new tasks types to Release:
 
@@ -47,12 +54,12 @@ The Mabl plugin adds the following new tasks types to Release:
 
 ### Run Test, Get Result
 
-The 'Run Test, Get Result' task will instruct Mabl to run the tests associated with the task configuration of 'Environment', 'Application' and 'Plan Labels'. The task will the wait for the test results and then poll the Mabl server, for as long as necessary.
+The 'Run Test, Get Result' task will instruct the configured Mabl Server Workspace to run the tests associated with the task configuration of 'Environment', 'Application' and 'Plan Labels'. The task will then wait for the test results by polling the Mabl server, for as long as necessary.
 ![mabl_runTestGetResult](images/runTestGetResult.png)
 
 ### Run Test
 
-The 'Run Test' task will instruct Mabl to run test that meet the configured criteria but will not wait for the results. Instead, the task will output a 'testId' value that can be stored in a Release variable. In this way, a future 'Get Result' task can retrieve the test result.
+The 'Run Test' task will instruct the configured Mabl Server Workspace to run tests that meet the configured criteria but will not wait for the results. Instead, the task will output a 'testId' value that can be stored in a Release variable. In this way, a future 'Get Result' task can retrieve the test result.
 ![mabl_runTest](images/runTest.png)
 
 Note: Both the 'Run Test, Get Results' task and the 'Run Test' task must have either or both of the 'Environment' and 'Application' fields populated. The 'Plan Labels' list is optional.
@@ -62,6 +69,62 @@ Note: Both the 'Run Test, Get Results' task and the 'Run Test' task must have ei
 Given a testId, the 'Get Result' task will poll Mabl to retrieve test results. 
 ![mabl_getResult](images/getResult.png)
 
+### Use a Value Provider to Set Environment, Application and Plan Labels
+
+Instead of manually entering the application, environment and/or plan labels, you can configure the 'Run Test, Get Result' or 'Run Test' tasks to use a Template Release Variable that will be pre-populated by a Value Provider. The Value Provider will query the configured Mabl Server Workspace to retrieve all of possible environment, application and plan label values. When a release is created you can then choice the specific values to assign to the Release Variables. Follow the steps below to create and use a Release Variable pre-populated by a Value Provider. 
+
+#### Create a Template Release Variable to Display All Possible Environments
+
+1. Create a Release Variable at the Template level of type List box
+2. This variable will be populated with environment ids from a single Mabl Server Workspace. It is a good practice name and label the variable so that the workspace association is clear.
+3. Choose the Value Provider type 'Value Provider'.
+4. From the Script Value Provider drop down list, choose Mabl Environments Value Provider.
+5. Choose the Mabl Server Workspace
+6. Make sure 'Show on Create Release Form' is checked. Do not check the 'Required' checkbox.
+7. Within the template, assign the variable you have just created to the task 'Environment Id' entry
+8. NOTE: Be sure your task and your Release Variable are both assigned to work within the same Mabl Server Workspace
+9. When you create the new release, you will be able to choose the specific Environment Id to use in the task
+![mabl_evnValueProv](images/evnValueProv.png)
+![mabl_envVariableAssign](images/envVariableAssign.png)
+
+#### Create a Template Release Variable to Display All Possible Applications
+
+1. Create a Release Variable at the Template level of type List box
+2. This variable will be populated with application ids from a single Mabl Server Workspace. It is a good practice name and label the variable so that the workspace association is clear.
+3. Choose the Value Provider type 'Value Provider'.
+4. From the Script Value Provider drop down list, choose Mabl Applications Value Provider.
+5. Choose the Mabl Server Workspace
+6. Make sure 'Show on Create Release Form' is checked. Do not check the 'Required' checkbox.
+7. Within the template, assign the variable you have just created to the task 'Application Id' entry.
+8. When you create the new release, you will be able to choose the specific Application Id to use in the task.
+![mabl_appValueProv](images/appValueProv.png)
+![mabl_appVariableAssign](images/appVariableAssign.png)
+
+#### Create a Template Release Variable to Display All Possible Plan Labels
+
+1. Create a Release Variable at the Template level of type Multi-Select List box. NOTE: This is a different type than the type used for the environment and application value providers.
+2. This variable will be populated with plan labels from a single Mabl Server Workspace. It is a good practice name and label the variable so that the workspace association is clear.
+3. Choose the Value Provider type 'Value Provider'.
+4. From the Script Value Provider drop down list, choose Mabl Plan Labels Value Provider.
+5. Choose the Mabl Server Workspace
+6. Make sure 'Show on Create Release Form' is checked. Do not check the 'Required' checkbox.
+7. Within the template, assign the variable you have just created to the task 'Plan Labels' entry. You will need to click the variable symbol displayed after the entry field labels so that you can choose the correct variable from the drop down list.
+8. When you create the new release, you will be able to choose the specific Plan Labels to use in the task.
+![mabl_planValueProv](images/planValueProv.png)
+![mabl_planVariableAssign](images/planVariableAssign.png)
+
+#### Tips for using Value Provider Release Variables
+
+When you create the new release, you will have an opportunity to choose the actual environment, application and plan labels. When the release is first created, the Value Provider scripts run and the workspace is queried for environment, application and plan labels. It may take a moment for the drop down lists and multi-select list to populate. When the queries are complete, the cursor will change from an arrow to a hand symbol and a drop down symbol will be displayed when you click the entry box. Choose a value from the drop down list. You can choose multiple Plan Label values. It is important that Mabl Server Workspace associated with the Variable and the Release Task are the same. If your Release has tasks that work with various Mabl Server Workspaces, create multiple Value Provider Release Variable associated with the various Workspaces. Be sure to label them clearly to prevent confusion.
+
+Note: The values for environments and applications include both the ids and the names. This is done for convenience. The plugin will process the string, parse out the id and send only the id to the Mabl Server Workspace.
+
+![mabl_createNewRelease](images/createNewRelease.png)
+![mabl_variableValues](images/variableValues.png)
+![mabl_planValues](images/planValues.png)
+
+
+
 ## Testing the plugin
 
 This project contains a docker-compose file and configuration yaml files so that you can spin up a version of Release with the Mabl plugin installed and then use the XL CLI to import configuration files to create a test scenario. You will need to create the src/test/resource/docker/initialize/yaml/xebialabs/secrets.xlvals file to hold the value of your API Token. You will also need to edit the xlr-configuration.yaml file to replace the environment, application and plan labels values with values that match your Mabl account configuration.

build.gradle

@@ -9,7 +9,7 @@ apply plugin: 'idea'
 apply plugin: 'eclipse'
 apply plugin: 'maven'
 
-version = "1.0.3"
+version = "1.1"
 
 if (!project.hasProperty('release.scope')) {
   project.ext['release.scope'] = 'patch'

src/main/resources/mabl/ApplicationsValueProvider.py

@@ -0,0 +1,24 @@
+#
+# Copyright 2022 DIGITAL.AI
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+
+import mabl.MablClient
+if classReload:
+    reload(mabl.MablClient)
+from mabl.MablClient import Mabl_Client
+import logging
+
+logger = logging.getLogger('mabl.'+'ApplicationsValueProvider')
+
+def findApplications():
+    mablObj = Mabl_Client.create_client(mablServer, None, tokenCICD)
+    retrievedApplicationsList = mablObj.mabl_getApplications(locals())
+    return retrievedApplicationsList
+
+result = findApplications()

src/main/resources/mabl/EnvironmentsValueProvider.py

@@ -0,0 +1,24 @@
+#
+# Copyright 2022 DIGITAL.AI
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+
+import mabl.MablClient
+if classReload:
+    reload(mabl.MablClient)
+from mabl.MablClient import Mabl_Client
+import logging
+
+logger = logging.getLogger('mabl.'+'EnvironmentsValueProvider')
+
+def findEnvironments():
+    mablObj = Mabl_Client.create_client(mablServer, None, tokenCICD)
+    retrievedEnvironmentsList = mablObj.mabl_getEnvironments(locals())
+    return retrievedEnvironmentsList
+
+result = findEnvironments()

src/main/resources/mabl/GetResult.py

@@ -14,6 +14,6 @@
 import logging
 
 logger = logging.getLogger('mabl.'+'GetResult')
-mablObj = Mabl_Client.create_client(mablServer, token)
+mablObj = Mabl_Client.create_client(mablServer, token, None, failTaskOnFailedTest)
 logger.debug("##### Start Polling ####")
 task.schedule("mabl/waitForTest.py", pollTime)

src/main/resources/mabl/MablClient.py

@@ -17,24 +17,30 @@
 from xlrelease.HttpRequest import HttpRequest
 import logging
 import requests
+import re
 
 logger = logging.getLogger(__name__)
 
 HTTP_SUCCESS = sets.Set([200, 201, 202, 203, 204, 205, 206, 207, 208])
 HTTP_ERROR = sets.Set([400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410,412, 413, 414, 415])
 
 class Mabl_Client(object):
-    def __init__(self, httpConnection, token=None):
+    def __init__(self, httpConnection, token=None, tokenCICD=None, failTaskOnFailedTest=True):
         self.httpConnection = httpConnection
+        self.workspaceId = httpConnection['workspaceId']
         self.token = token
         if token is None:
             self.token = httpConnection['token']
+        self.tokenCICD = tokenCICD
+        if tokenCICD is None:
+            self.tokenCICD = httpConnection['tokenCICD']
+        self.failTaskOnFailedTest = failTaskOnFailedTest
         self.httpRequest = HttpRequest(httpConnection)
 
 
     @staticmethod
-    def create_client(httpConnection, token=None):
-        return Mabl_Client(httpConnection, token)
+    def create_client(httpConnection, token=None, tokenCICD=None, failTaskOnFailedTest=True):
+        return Mabl_Client(httpConnection, token, tokenCICD, failTaskOnFailedTest)
 
 
     def testServer(self):
@@ -64,10 +70,10 @@ def mabl_runtest(self, variables):
         payload =   {}
 
         if len(variables['environmentId']) > 0:
-            env = {"environment_id": variables['environmentId']}
+            env = {"environment_id": self.parseEnv(variables['environmentId'])}
             payload.update(env)
         if len(variables['applicationId']) > 0:
-            app = {"application_id": variables['applicationId']}
+            app = {"application_id": self.parseApp(variables['applicationId'])}
             payload.update(app)
         if len(variables['planLabels']) > 0:
             plans = {"plan_labels": variables['planLabels']}
@@ -87,6 +93,21 @@ def mabl_runtest(self, variables):
         data = json.loads(response.getResponse())
         return data['id']
 
+    def parseEnv(self, envString):
+        regex = r"[a-zA-Z0-9]*-e"
+        if re.search(regex, envString):
+            return re.search(regex, envString).group()
+        else:
+            raise Exception("The entered string - %s - does not contain an environment id. Expected pattern is alpha-numeric followed by -e." % envString)
+
+    def parseApp(self, appString):
+        regex = r"[a-zA-Z0-9]*-a"
+        if re.search(regex, appString):
+            return re.search(regex, appString).group()
+        else:
+            raise Exception("The entered string - %s - does not contain an application id. Expected pattern is alpha-numeric followed by -a." % appString)
+
+
 
     def mabl_waitfortest(self, variables):
         api_key_bytes = self.token.encode('ascii')
@@ -127,7 +148,89 @@ def mabl_waitfortest(self, variables):
         data['output']['journeyUrl'] = journeyUrls
         return data
 
+    def mabl_getEnvironments(self, variables):
+        api_key_bytes = self.tokenCICD.encode('ascii')
+        Base64String = base64.b64encode(api_key_bytes)
+        Base64String = "Basic %s" % Base64String
+
+        headers = {
+          "Content-Type": "application/json",
+          "Authorization": Base64String
+        }
+
+        mablUrl = "environments?organization_id=%s" % self.workspaceId
+        logger.debug("Requesting Environments %s" % mablUrl)
+        response = self.httpRequest.get(mablUrl, headers=headers, contentType='application/json')
+        data = json.loads(response.getResponse())
+        logger.debug("Get Environments Response\n=============\n%s\n==================" % json.dumps(data, indent=4, sort_keys=True))
+
+        if response.getStatus() not in HTTP_SUCCESS:
+            logger.error("Get Environments Request Error (%s)" % response.getStatus())
+            self.throw_error(response)
+
+        data = json.loads(response.getResponse())
+        environmentIds = []
+        for env in data['environments']:
+            environmentIds.append("%s (%s)" % (env["id"], env["name"]))
+        return environmentIds
+
+    def mabl_getApplications(self, variables):
+        api_key_bytes = self.tokenCICD.encode('ascii')
+        Base64String = base64.b64encode(api_key_bytes)
+        Base64String = "Basic %s" % Base64String
+
+        headers = {
+          "Content-Type": "application/json",
+          "Authorization": Base64String
+        }
+
+        mablUrl = "applications?organization_id=%s" % self.workspaceId
+        logger.debug("Requesting Applications %s" % mablUrl)
+        response = self.httpRequest.get(mablUrl, headers=headers, contentType='application/json')
+        data = json.loads(response.getResponse())
+        logger.debug("Get Environments Response\n=============\n%s\n==================" % json.dumps(data, indent=4, sort_keys=True))
+
+        if response.getStatus() not in HTTP_SUCCESS:
+            logger.error("Get Applications Request Error (%s)" % response.getStatus())
+            self.throw_error(response)
+
+        data = json.loads(response.getResponse())
+        applicationIds = []
+        for app in data['applications']:
+            applicationIds.append("%s (%s)" % (app["id"], app["name"]))
+        return applicationIds
+
+    def mabl_getLabels(self, variables):
+        api_key_bytes = self.tokenCICD.encode('ascii')
+        Base64String = base64.b64encode(api_key_bytes)
+        Base64String = "Basic %s" % Base64String
+
+        headers = {
+          "Content-Type": "application/json",
+          "Authorization": Base64String
+        }
+
+        mablUrl = "schedule/runPolicy/labels?organization_id=%s" % self.workspaceId
+        logger.debug("Requesting Labels %s" % mablUrl)
+        response = self.httpRequest.get(mablUrl, headers=headers, contentType='application/json')
+        data = json.loads(response.getResponse())
+        logger.debug("Get Labels Response\n=============\n%s\n==================" % json.dumps(data, indent=4, sort_keys=True))
+
+        if response.getStatus() not in HTTP_SUCCESS:
+            logger.error("Get Labels Request Error (%s)" % response.getStatus())
+            self.throw_error(response)
+
+        data = json.loads(response.getResponse())
+        labelNames = []
+        for lab in data['labels']:
+            labelNames.append(lab["name"])
+        return labelNames
+
 
     def throw_error(self, response):
         logger.error("Error from MablService, HTTP Return: %s\n" % ( response.getStatus() ) )
-        raise Exception(response.getStatus())
+        message = ""
+        if response.getResponse():
+            logger.error("Error from MablService, HTTP Return message: %s\n" % ( response.getResponse() ) )
+            message = response.getResponse()
+        raise Exception("Response status: %s, response message: %s " % (str(response.getStatus()), message))