Skip to content

Latest commit

 

History

History
354 lines (258 loc) · 24.4 KB

howto-update-image.md

File metadata and controls

354 lines (258 loc) · 24.4 KB
Raw

GitHub Actions automation

This automation aims to simplify the process for keeping the Partner Center offers for WebSphere traditional up to date, with the latest fix packs, iFixes, yum updates, etc. At a high level, the least-effort update involves the following steps.

  • Incrementing the versions in several pom.xml files.
  • Updating a property in a few JSON files.
  • Running a few GitHub Actions workflows.
  • Selecting a few buttons in Partner Center.

This document describes two different approaches for causing an update to the code in this repository to be packaged, published and made available to users in Partner Center.

  • The automated process
  • The manual process.

Use of the processes is not mutually exclusive. Both processes share code to avoid DRY violations. The shared code does a number of "software update" type actions such as yum update, installing iFixes, etc.

IMPORTANT NOTES

  • Both of these processes include polling software repositories for operating system, JDK, and middleware updates. If you are looking to "pick up" a recently release update, iFix, or other by virtue of running through the process, please be sure the particular item you are looking to "pick up" is present in the corresponding repository before running the process.

  • Don't put any comments in any JSON files. Even if Microsoft documentation says it is safe to do so.

  • Although it is tempting to try to parallelize some of these steps, in practice this is impossible due to dependencies on Azure Marketplace publishing outcomes. The authors advise doing the steps entirely in serial.

The automated process

The GitHub Actions workflows in this repository, and the related repositories referenced in the top level README, automate the process of packaging, testing, and publishing Azure virtual machine and Azure Application offers to Microsoft Partner Center. The automated process does involve a small amount of manual action in Partner Center, as described below.

Preconditions

Both sections of preconditions must be satisfied before running the workflows. [expand for details]

Once the preconditions have been satisfied, they remain satisfied unless and until a change is made that would invalidate the set up in some way.

1. Set the GitHub Actions secrets for the repository running the workflows

The recommended way to set the secrets is to run the scripts. Setting the secrets manually is beyond the scope of this guide.

Preconditions for running the scripts to set the secrets
  1. Ensure the Azure CLI is installed on a supported UNIX-like environment. See How to install the Azure CLI. Sign in to Azure using the Azure CLI. After installation, sign in to the correct tenant. The actions will create Azure resources in this signed-in tenant.
  2. Ensure the GitHub CLI is installed on the same environment as the preceding step. See Installation. Note: If working on macOS, we highly recommend Homebrew. Visit https://brew.sh/ for instructions on installing Homebrew. Authenticate to GitHub. After installation, use gh auth login to sign in to GitHub. You'll need a sufficiently empowered PERSONAL ACCESS TOKEN for this repository.
  3. Ensure yq 4.x installed. See yq for installation instructions.
  4. Clone this repository into the environment from the preceding steps.
Setting the secrets
  • Follow guidance on top of setup-credentials.sh to run the shell scripts. The scripts will set the necessary secrets for the repository running the workflows.
  • If you want to tear down the secrets, follow the guidance on top of teardown-credentials.sh to run the shell scripts. The scripts will remove the necessary secrets for the repository running the workflows.

2. Set the GitHub Actions secrets for the related repositories

The related repositories referenced in the top level README have analogous scripts to set up and tear down credentials. The preconditions and invocation for these scripts are the same as in the preceding section.

Running the workflows and publishing the offers

Now that you have satisfied the preconditions in this repository and related repositories, you can run the workflows.

Running the workflows and publishing the offers for WebSphere Application Server traditional Base

The steps in this section describe how to run the workflows and publish the VM offer and Azure Application offer for WebSphere Application Server traditional Base. [expand for details]

1. Increment the version of tWAS Base VM offer in the pom.xml

  1. Increment the version of twas-base/pom.xml.
  2. Push the commit to the branch on which you intend to run the workflow in the next step.

2. Run the workflow for the tWAS Base VM offer

Run the workflow to create the tWAS Base VM offer. [expand for details]
  1. Decide on a value for the imageVersionNumber parameter. The required syntax for this value is 9.0.YYYYMMDD. Where YYYYMMDD is usually today's date.
  2. Visit the GitHub Actions page for the twas-base CICD workflow.
  3. Select the Run workflow dropdown. Enter the value for imageVersionNumber under Provide image version number
    • Please note that the imageVersionNumber must be provided if you want to publish the image to Partner Center.
    • If you do not provide imageVersionNumber value, the workflow will only used for test purpose.
  4. Select Run workflow.
  5. Observe the execution of the jobs in the workflow.
    • One very important job is Verify the image. This job calls another workflow, on the related repository for the Azure Application, but the VM image created by the calling workflow is taken as input to this called workflow.

If the workflow completes successfully, proceed to the next section. If not, troubleshoot and resolve the problem with guidance from the section on The manual process before proceeding.

3. Publish the tWAS Base VM offer in Partner Center

Because the workflow in the preceding section executed successfully, you can assume the VM image is ready to publish in Partner Center.

Use Partner Center to publish the VM offer and track to live. [expand for details]
  1. Visit Partner Center at https://partner.microsoft.com/.
  2. Sign in to the partner center by selecting the Partner Center link in the upper right corner of the page, next to Search. You must sign in this way.
  3. Select Marketplace offers.
  4. In the textfield labeled Search by offer alias and ID, enter 2023-03-27-twas-single-server-base-image.
  5. Select the one and only row. If you see more than one row, consult with management to see which one to select.
  6. In the left navigation panel, select Plan overview.
  7. On the next page, select the one and only plan.
  8. On the next page, in the left navigation panel, select Technical configuration.
  9. In the VM Images section, you should see a row whose Image version column is the same as the value of imageVersionNumber you entered previously. If you do not see this value, troubleshoot and resolve the problem with guidance from the section on The manual process before proceeding.
  10. The previously run workflow will have updated the technical configuration. Go to the bottom of the page and select Review and publish.
  11. On the next page, in the text area, paste the URL to the successful GitHub Actions workflow from the preceding section.
  12. Select Publish.
  13. This should take you back to the Offer overview page, but the progress bar will now be partially filled in.
  14. The offer will go through the publishing process.
  15. After some hours or maybe days, the offer will enter "preview" state. In this state, you can manually test the offer. The CI/CD has already done sufficient testing, but you can do more if you like.
  16. Select the big Go Live button.
  17. After some hours, or maybe days, the offer will enter "live" state.

4. Update the source files in the tWAS Base (aka single-server) Azure Application offer

At this point, the tWAS Base Azure VM offer is live. This same VM offer has been tested with the Azure Application offer, but the source code changes to publish a new iteration of the Azure Application offer have not been updated.

The steps is this section show how to update the source files to use the new VM offer.
  1. Visit the https://github.com/WASdev/azure.websphere-traditional.singleserver repository.
  2. Increment the version of pom.xml.
  3. If creating a new Plan, update the pid value as described in How Azure customer usage attribution works in the IBM Partner Center offers.
  4. Edit src/main/bicep/config.json.
    1. Change the value of twasImageVersion to be the value entered for imageVersionNumber previously.
  5. Push the commit to the branch on which you intend to run the workflow in the next step.

5. Run the workflow for the tWAS Base (aka single-server) Azure Application offer

You can publish the corresponding Azure Application offer that uses that base image.

Run the workflow to create the tWAS Base (aka single-server) Azure Application offer. [expand for details]
  1. Visit the https://github.com/WASdev/azure.websphere-traditional.singleserver repository.
  2. In that repository, select the Actions tab.
  3. Select the Package ARM workflow.
  4. Select the Run workflow dropdown.
  5. Select Run workflow.
  6. Observe the execution of the jobs in the workflow.

If the workflow completes successfully, proceed to the next section. If not, troubleshoot and resolve the problem with guidance from the section on The manual process before proceeding.

6. Publish the tWAS Base Azure Application offer in Partner Center

Because the workflow in the preceding section executed successfully, you can assume the Azure Application offer is ready to publish in Partner Center.

Use Partner Center to publish the Azure Application offer and track to live. [expand for details]
  1. Visit Partner Center at https://partner.microsoft.com/.
  2. Sign in to the partner center by selecting the Partner Center link in the upper right corner of the page, next to Search. You must sign in this way.
  3. Select Marketplace offers.
  4. In the textfield labeled Search by offer alias and ID, enter 2022-01-07-twas-base-single-server.
  5. Select the one and only row. If you see more than one row, consult with management to see which one to select.
  6. In the left navigation panel, select Plan overview.
  7. On the next page, select the one and only plan.
  8. On the next page, in the left navigation panel, select Technical configuration.
  9. The previously run workflow will have updated the technical configuration. Select Review and publish.
  10. On the next page, in the text area, paste the URL to the successful GitHub Actions workflow from the preceding section.
  11. Select Publish.
  12. This should take you back to the Offer overview page, but the progress bar will now be partially filled in.
  13. The offer will go through the publishing process.
  14. After some hours or maybe days, the offer will enter "preview" state. In this state, you can manually test the offer. The CI/CD has already done sufficient testing, but you can do more if you like.
  15. Select the big Go Live button.
  16. After some hours, or maybe days, the offer will enter "live" state.

7. Cleanup test resources

https://github.com/WASdev/azure.websphere-traditional.image/blob/main/docs/howto-cleanup-after-image-published.md#find-out-the-storage-account-from-azure-portal

Running the workflows for WebSphere Application Server traditional Network Deployment

The steps in this section describe how to run the workflows and publish the VM offers and Azure Application offer for WebSphere Application Server ND [expand for details]
The steps in this subsection show how to run the workflows and publish the VM offers. [expand for details]

1. Increment the version of ihs VM offer in the pom.xml

  1. Increment the version of ihs/pom.xml.
  2. Push the commit to the branch on which you intend to run the workflow in subsequent steps.

2. Increment the version of tWAS ND VM offer in the pom.xml

  1. Increment the version of twas-nd/pom.xml.
  2. Push the commit to the branch on which you intend to run the workflow in subsequent steps.

3. Run the workflow for the ihs VM offer

Run the workflow to create the IHS VM offer. [expand for details]
  1. Decide on a value for the imageVersionNumber parameter. The required syntax for this value is 9.0.YYYYMMDD. Where YYYYMMDD is usually today's date.
  2. Visit the GitHub Actions page for the workflow.
  3. The remaining steps are the same as in the section Run the workflow for the tWAS Base VM offer.

If the workflow completes successfully, proceed to the next section. If not, troubleshoot and resolve the problem with guidance from the section on The manual process before proceeding.

4. Run the workflow for the tWAS ND VM offer

Run the workflow to create the tWAS ND VM offer. [expand for details]
  1. Decide on a value for the imageVersionNumber parameter. The required syntax for this value is 9.0.YYYYMMDD. Where YYYYMMDD is usually today's date.
  2. Visit the GitHub Actions page for the workflow.
  3. The remaining steps are the same as in the section Run the workflow for the tWAS Base VM offer.

If the workflow completes successfully, proceed to the next section. If not, troubleshoot and resolve the problem with guidance from the section on The manual process before proceeding.

5. Publish the offers in Partner Center

Because the workflows in the preceding sections executed successfully, you can assume the VM images are ready to publish in Partner Center.

Use Partner Center to publish the VM offers for IHS and tWAS ND and track to live. [expand for details]
  1. Visit Partner Center at https://partner.microsoft.com/.
  2. Sign in to the partner center by selecting the Partner Center link in the upper right corner of the page, next to Search. You must sign in this way.
  3. Select Marketplace offers.
  4. In the textfield labeled Search by offer alias and ID, enter 2023-03-27-ihs-base-image.
  5. Select the one and only row. If you see more than one row, consult with management to see which one to select.
  6. In the left navigation panel, select Plan overview.
  7. On the next page, select the one and only plan.
  8. On the next page, in the left navigation panel, select Technical configuration.
  9. In the VM Images section, you should see a row whose Image version column is the same as the value of imageVersionNumber you entered previously for IHS and for tWAS ND. If you do not see this value, troubleshoot and resolve the problem with guidance from the section on The manual process before proceeding.
  10. The previously run workflow will have updated the technical configuration. Go to the bottom of the page and select Review and publish.
  11. On the next page, in the text area, paste the URL to the successful GitHub Actions workflow from the preceding section.
  12. Select Publish.
  13. This should take you back to the Offer overview page, but the progress bar will now be partially filled in.
  14. The offer will go through the publishing process.
  15. Return to the Marketplace offers | Overview page by selecting Marketplace offers in the breadcrumb navigation in the top left of the screen.
  16. Return to step 4, but enter 2023-03-27-twas-cluster-base-image in the textfield. Continue the remaining steps up to and including 14. Be sure to use the correct imageVersionNumber value for tWAS ND.
  17. After some hours or maybe days, the offers will enter "preview" state. In this state, you can manually test the offer. The CI/CD has already done sufficient testing, but you can do more if you like.
  18. For each of the two offers you published previously, select the big Go Live button.
  19. After some hours, or maybe days, the offer will enter "live" state.
The steps in this subsection show how to run the workflows and publish the Azure Application offer that uses the VM offers from the preceding section. [expand for details]

6. Update the source files in the tWAS ND (aka cluster) Azure Application offer

At this point, the tWAS ND and IHS Azure VM offers are live. These same VM offers have already been tested via CI/CD with the Azure Application offer, but the source code changes to publish a new iteration of the Azure Application offer have not been updated.

The steps is this section show how to update the source files to use the new VM offer.
  1. Visit the https://github.com/WASdev/azure.websphere-traditional.cluster repository.
  2. Increment the version of pom.xml.
  3. If creating a new Plan, update the pid value as described in How Azure customer usage attribution works in the IBM Partner Center offers.
  4. Edit src/main/bicep/config.json.
    1. Change the value of ihsImageVersion to be the value entered for imageVersionNumber when you created the IHS image previously.
    2. Change the value of twasNdImageVersion to be the value entered for imageVersionNumber when you created the tWAS ND image previously.
  5. Push the commit to the branch on which you intend to run the workflow in the next step.

7. Run the workflow for the tWAS ND (aka cluster) Azure Application offer

You can publish the corresponding Azure Application offer that uses the base images for IHS and tWAS ND.

Run the workflow to create the tWAS ND (aka cluster) Azure Application offer. [expand for details]
  1. Visit the https://github.com/WASdev/azure.websphere-traditional.cluster repository.
  2. In that repository, select the Actions tab.
  3. Select the Package ARM workflow.
  4. Select the Run workflow dropdown.
  5. Select Run workflow.
  6. Observe the execution of the jobs in the workflow.

If the workflow completes successfully, proceed to the next section. If not, troubleshoot and resolve the problem with guidance from the section on The manual process before proceeding.

8. Publish the tWAS ND Azure Application offer in Partner Center

Because the workflow in the preceding section executed successfully, you can assume the Azure Application offer is ready to publish in Partner Center.

Use Partner Center to publish the Azure Application offer and track to live. [expand for details]
  1. Visit Partner Center at https://partner.microsoft.com/.
  2. Sign in to the partner center by selecting the Partner Center link in the upper right corner of the page, next to Search. You must sign in this way.
  3. Select Marketplace offers.
  4. In the textfield labeled Search by offer alias and ID, enter 2021-04-08-twas-cluster.
  5. There may be more than one row. Select the one in the whose Offer type is Azure Application. If you see more than one row with type Azure Application, consult with management to see which one to select.
  6. In the left navigation panel, select Plan overview.
  7. On the next page, select the one and only plan.
  8. On the next page, in the left navigation panel, select Technical configuration.
  9. The previously run workflow will have updated the technical configuration. Select Review and publish.
  10. On the next page, in the text area, paste the URL to the successful GitHub Actions workflow from the preceding section.
  11. Select Publish.
  12. This should take you back to the Offer overview page, but the progress bar will now be partially filled in.
  13. The offer will go through the publishing process.
  14. After some hours or maybe days, the offer will enter "preview" state. In this state, you can manually test the offer. The CI/CD has already done sufficient testing, but you can do more if you like.
  15. Select the big Go Live button.
  16. After some hours, or maybe days, the offer will enter "live" state.

9. Cleanup test resources

https://github.com/WASdev/azure.websphere-traditional.image/blob/main/docs/howto-cleanup-after-image-published.md#find-out-the-storage-account-from-azure-portal

The manual process

See these links for guidance on how to update VM images.

  1. For image twas-base, see this.
  2. For images twas-nd and ihs, see this.

Troubleshooting

  • CICD build fails with Api versions must be the latest or under 2 years old
  • CICD build fails with Could not find artifact com.microsoft.azure.iaas:azure-javaee-iaas-parent:pom:1.0.18
  • Partner Center publishing failes for VM base image: Attention needed
    • This might cause the offer to be left in a Pending publish state. When an offer is left in this state, any subsequent attempts to publish a new base image will fail. Follow these steps to remove the image left in the Pending publish state.
      1. Sign in to Partner Center.
      2. Select Marketplace offers.
      3. Find the offer that failed. It will likely show Attention needed in the Status column.
      4. Select that offer.
      5. In the left navigation pane, select Plan overview.
      6. Select the one and only one row in the table.
      7. In the left navigation pane, select Technical configuration.
      8. In the VM Images section, find the table. The bottom most row will likely show Pending publish in status. Select the cell for that row in the Image version column.
      9. In the SAS URI section, select Remove for all items in that section.
      10. Select Save VM image.
      11. You will be taken to the VM Images table again. The bottom most row will now show No actions available in the Action column.
      12. In the left navigation pane, select Plan overview.
      13. Select the one and only one row in the table.
      14. In the left navigation pane, select Technical configuration.
      15. In the VM Images section, find the table.
      16. The bottom most row should now have Delete image in the Action column. Select Delete image.
      17. Select Confirm.
      18. Scroll to the bottom and select Save draft.
      19. You should see a green bar notification toward the top of the page. If you see this, you now should be able to publish a new image.