feat: Root module updates:<br>- existing_kms_instance_guid is no lo…

…nger a supported input. The code will now parse the GUID from the KMS key CRN<br>- added new input `use_same_kms_key_for_backups` to give more control over KMS key usage<br>- `kms_encryption_enabled` has been renamed to `use_ibm_owned_encryption_key`<br>- `fscloud` submodule updates:<br>- added new inputs `use_default_backup_encryption_key` and `use_same_kms_key_for_backups`<br>DA updates<br>- Removed the input `existing_backup_kms_instance_crn`. If you want to use a different KMS key for backups, you can use the `existing_backup_kms_key_crn` input to use an existing key. The DA only supports creating an new key that will be used for both data and backups encryption.<br>- The `skip_iam_authorization_policy` input has been renamed to `skip_es_kms_auth_policy`<br> If passing a value for `ibmcloud_kms_api_key`, and creating a KMS auth policy, that policy will now be scoped to the exact KMS key. (#351)

.catalog-onboard-pipeline.yaml

@@ -12,3 +12,9 @@ offerings:
         scc:
           instance_id: 1c7d5f78-9262-44c3-b779-b28fe4d88c37
           region: us-south
+      - name: enterprise
+        mark_ready: true
+        install_type: fullstack
+        scc:
+          instance_id: 1c7d5f78-9262-44c3-b779-b28fe4d88c37
+          region: us-south

README.md

@@ -38,10 +38,10 @@ unless real values don't help users know what to change.
 ```hcl
 module "event_streams" {
   source  = "terraform-ibm-modules/event-streams/ibm"
-  version = "latest" # Replace "latest" with a release version to lock into a specific release
-  resource_group    = "event-streams-rg"
-  plan                 = "standard"
-  topics           = [
+  version = "X.Y.Z" # Replace "X.Y.Z" with a release version to lock into a specific release
+  resource_group_id = "xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX" # Replace with the actual ID of resource group to use
+  plan              = "standard"
+  topics            = [
     {
       name       = "topic-1"
       partitions = 1
@@ -63,7 +63,7 @@ module "event_streams" {
       }
     }
   ]
-  schema_id            = [{
+  schemas            = [{
     schema_id = "my-es-schema_1"
     schema = {
       type = "string"
@@ -115,6 +115,7 @@ You need the following permissions to run this module.
 |------|--------|---------|
 | <a name="module_cbr_rule"></a> [cbr\_rule](#module\_cbr\_rule) | terraform-ibm-modules/cbr/ibm//modules/cbr-rule-module | 1.29.0 |
 | <a name="module_es_guid_crn_parser"></a> [es\_guid\_crn\_parser](#module\_es\_guid\_crn\_parser) | terraform-ibm-modules/common-utilities/ibm//modules/crn-parser | 1.1.0 |
+| <a name="module_kms_key_crn_parser"></a> [kms\_key\_crn\_parser](#module\_kms\_key\_crn\_parser) | terraform-ibm-modules/common-utilities/ibm//modules/crn-parser | 1.1.0 |
 
 ### Resources
 
@@ -143,7 +144,6 @@ You need the following permissions to run this module.
 | <a name="input_create_timeout"></a> [create\_timeout](#input\_create\_timeout) | The timeout value for creating an Event Streams instance. Specify `3h` for an Enterprise plan instance. Add 1 h for each level of non-default throughput. Add 30 min for each level of non-default storage size. | `string` | `"3h"` | no |
 | <a name="input_delete_timeout"></a> [delete\_timeout](#input\_delete\_timeout) | The timeout value for deleting an Event Streams instance. | `string` | `"15m"` | no |
 | <a name="input_es_name"></a> [es\_name](#input\_es\_name) | The name to give the Event Streams instance created by this module. | `string` | n/a | yes |
-| <a name="input_existing_kms_instance_guid"></a> [existing\_kms\_instance\_guid](#input\_existing\_kms\_instance\_guid) | The GUID of the Hyper Protect Crypto Services or Key Protect instance in which the key specified in var.kms\_key\_crn is coming from. Required only if var.kms\_encryption\_enabled is set to true, var.skip\_kms\_iam\_authorization\_policy is set to false, and you pass a value for var.kms\_key\_crn. | `string` | `null` | no |
 | <a name="input_kms_encryption_enabled"></a> [kms\_encryption\_enabled](#input\_kms\_encryption\_enabled) | Set this to true to control the encryption keys used to encrypt the data that you store in IBM Cloud® Databases. If set to false, the data is encrypted by using randomly generated keys. For more info on Key Protect integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-key-protect. For more info on HPCS integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs | `bool` | `false` | no |
 | <a name="input_kms_key_crn"></a> [kms\_key\_crn](#input\_kms\_key\_crn) | The root key CRN of the key management service (Key Protect or Hyper Protect Crypto Services) to use to encrypt the payload data. [Learn more](https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-managing_encryption) about integrating Hyper Protect Crypto Services with Event Streams. | `string` | `null` | no |
 | <a name="input_metrics"></a> [metrics](#input\_metrics) | Enhanced metrics to activate, as list of strings. Only allowed for enterprise plans. Allowed values: 'topic', 'partition', 'consumers'. | `list(string)` | `[]` | no |
@@ -158,7 +158,7 @@ You need the following permissions to run this module.
 | <a name="input_service_credential_names"></a> [service\_credential\_names](#input\_service\_credential\_names) | The mapping of names and roles for service credentials that you want to create for the Event streams. | `map(string)` | `{}` | no |
 | <a name="input_service_endpoints"></a> [service\_endpoints](#input\_service\_endpoints) | The type of service endpoints. Possible values: 'public', 'private', 'public-and-private'. | `string` | `"public"` | no |
 | <a name="input_skip_es_s2s_iam_authorization_policy"></a> [skip\_es\_s2s\_iam\_authorization\_policy](#input\_skip\_es\_s2s\_iam\_authorization\_policy) | Set to true to skip the creation of an IAM authorization policy that will allow all Event Streams instances in the given resource group access to read from the mirror source instance. This policy is required when creating a mirroring instance, and will only be created if a value is passed in the mirroring input. | `bool` | `false` | no |
-| <a name="input_skip_kms_iam_authorization_policy"></a> [skip\_kms\_iam\_authorization\_policy](#input\_skip\_kms\_iam\_authorization\_policy) | Set to true to skip the creation of an IAM authorization policy that permits all Event Streams database instances in the resource group to read the encryption key from the KMS instance. If set to false, pass in a value for the KMS instance in the `existing_kms_instance_guid` variable. In addition, no policy is created if var.kms\_encryption\_enabled is set to false. | `bool` | `false` | no |
+| <a name="input_skip_kms_iam_authorization_policy"></a> [skip\_kms\_iam\_authorization\_policy](#input\_skip\_kms\_iam\_authorization\_policy) | Set to true to skip the creation of an IAM authorization policy that permits all Event Streams database instances in the resource group to read the encryption key from the KMS instance. If set to false, pass in a value for the KMS instance in the `kms_key_crn` variable. In addition, no policy is created if var.kms\_encryption\_enabled is set to false. | `bool` | `false` | no |
 | <a name="input_storage_size"></a> [storage\_size](#input\_storage\_size) | Storage size of the Event Streams in GB. Applies only to Enterprise plan instances. Possible values: `2048`, `4096`, `6144`, `8192`, `10240`, `12288`. Storage capacity cannot be reduced after the instance is created. When the `throughput` input variable is set to `300`, storage size starts at 4096. When `throughput` is `450`, storage size starts starts at `6144`. | `number` | `"2048"` | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | The list of tags associated with the Event Steams instance. | `list(string)` | `[]` | no |
 | <a name="input_throughput"></a> [throughput](#input\_throughput) | Throughput capacity in MB per second. Applies only to Enterprise plan instances. Possible values: `150`, `300`, `450`. | `number` | `"150"` | no |

cra-config.yaml

@@ -1,9 +1,12 @@
 # More info about this file at https://github.com/terraform-ibm-modules/common-pipeline-assets/blob/main/.github/workflows/terraform-test-pipeline.md#cra-config-yaml
 version: "v1"
 CRA_TARGETS:
-  - CRA_TARGET: "examples/complete" # Target directory for CRA scan. If not provided, the CRA Scan will not be run.
+  - CRA_TARGET: "solutions/enterprise" # Target directory for CRA scan. If not provided, the CRA Scan will not be run.
     CRA_IGNORE_RULES_FILE: "cra-tf-validate-ignore-rules.json" # CRA Ignore file to use. If not provided, it checks the repo root directory for `cra-tf-validate-ignore-rules.json`
     PROFILE_ID: "fe96bd4d-9b37-40f2-b39f-a62760e326a3"         # SCC profile ID (currently set to 'IBM Cloud Framework for Financial Services' '1.7.0' profile).
     CRA_ENVIRONMENT_VARIABLES:
-      TF_VAR_existing_kms_instance_guid: "e6dce284-e80f-46e1-a3c1-830f7adff7a9"
-      TF_VAR_kms_key_crn: "crn:v1:bluemix:public:hs-crypto:us-south:a/abac0df06b644a9cabc6e44f55b3880e:e6dce284-e80f-46e1-a3c1-830f7adff7a9:key:76170fae-4e0c-48c3-8ebe-326059ebb533"
+      TF_VAR_existing_kms_instance_crn: "crn:v1:bluemix:public:hs-crypto:us-south:a/abac0df06b644a9cabc6e44f55b3880e:e6dce284-e80f-46e1-a3c1-830f7adff7a9::"
+      TF_VAR_resource_group_name: "test-event-s-cra"
+      TF_VAR_provider_visibility: "public"
+      TF_VAR_use_existing_resource_group: false
+      TF_VAR_kms_endpoint_type: "public"

examples/fscloud/main.tf

@@ -69,16 +69,15 @@ module "cbr_zone_schematics" {
 # #############################################################################
 
 module "event_streams" {
-  source                     = "../../modules/fscloud"
-  resource_group_id          = module.resource_group.resource_group_id
-  es_name                    = "${var.prefix}-es-fs"
-  kms_key_crn                = var.kms_key_crn
-  schemas                    = var.schemas
-  tags                       = var.resource_tags
-  topics                     = var.topics
-  existing_kms_instance_guid = var.existing_kms_instance_guid
-  metrics                    = ["topic", "partition", "consumers"]
-  mirroring_topic_patterns   = ["topic-1", "topic-2"]
+  source                   = "../../modules/fscloud"
+  resource_group_id        = module.resource_group.resource_group_id
+  es_name                  = "${var.prefix}-es-fs"
+  kms_key_crn              = var.kms_key_crn
+  schemas                  = var.schemas
+  tags                     = var.resource_tags
+  topics                   = var.topics
+  metrics                  = ["topic", "partition", "consumers"]
+  mirroring_topic_patterns = ["topic-1", "topic-2"]
   mirroring = {
     source_crn   = var.event_streams_source_crn # Required for mirroring
     source_alias = "source-alias"               # Required for mirroring

examples/fscloud/variables.tf

@@ -54,11 +54,6 @@ variable "topics" {
   default     = []
 }
 
-variable "existing_kms_instance_guid" {
-  description = "The GUID of the Hyper Protect Crypto service in which the key specified in var.kms_key_crn is coming from"
-  type        = string
-}
-
 variable "kms_key_crn" {
   type        = string
   description = "The root key CRN of a Hyper Protect Crypto Service (HPCS) that you want to use for disk encryption. See https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs&interface=ui for more information on integrating HPCS with Event Streams instance."

ibm_catalog.json

@@ -46,7 +46,25 @@
               "key": "ibmcloud_api_key"
             },
             {
-              "key": "prefix"
+              "key": "provider_visibility",
+              "options": [
+                {
+                  "displayname": "private",
+                  "value": "private"
+                },
+                {
+                  "displayname": "public",
+                  "value": "public"
+                },
+                {
+                  "displayname": "public-and-private",
+                  "value": "public-and-private"
+                }
+              ]
+            },
+            {
+              "key": "prefix",
+              "required": true
             },
             {
               "key": "use_existing_resource_group"
@@ -128,6 +146,9 @@
             },
             {
               "key": "topics"
+            },
+            {
+              "key": "service_credential_names"
             }
           ],
           "iam_permissions": [
@@ -167,6 +188,215 @@
               }
             ]
           }
+        },
+        {
+          "label": "enterprise",
+          "name": "enterprise",
+          "install_type": "fullstack",
+          "working_directory": "solutions/enterprise",
+          "configuration": [
+            {
+              "key": "ibmcloud_api_key"
+            },
+            {
+              "key": "provider_visibility",
+              "options": [
+                {
+                  "displayname": "private",
+                  "value": "private"
+                },
+                {
+                  "displayname": "public",
+                  "value": "public"
+                },
+                {
+                  "displayname": "public-and-private",
+                  "value": "public-and-private"
+                }
+              ]
+            },
+            {
+              "key": "prefix",
+              "required": true
+            },
+            {
+              "key": "use_existing_resource_group"
+            },
+            {
+              "key": "resource_group_name"
+            },
+            {
+              "key": "event_streams_name"
+            },
+            {
+              "key": "region",
+              "required": true,
+              "options": [
+                {
+                  "displayname": "Dallas (us-south)",
+                  "value": "us-south"
+                },
+                {
+                  "displayname": "Frankfurt (eu-de)",
+                  "value": "eu-de"
+                },
+                {
+                  "displayname": "London (eu-gb)",
+                  "value": "eu-gb"
+                },
+                {
+                  "displayname": "Madrid (eu-es)",
+                  "value": "eu-es"
+                },
+                {
+                  "displayname": "Osaka (jp-osa)",
+                  "value": "jp-osa"
+                },
+                {
+                  "displayname": "Sydney (au-syd)",
+                  "value": "au-syd"
+                },
+                {
+                  "displayname": "Tokyo (jp-tok)",
+                  "value": "jp-tok"
+                },
+                {
+                  "displayname": "Osaka (jp-osa)",
+                  "value": "jp-osa"
+                },
+                {
+                  "displayname": "Sao Paulo (br-sao)",
+                  "value": "br-sao"
+                },
+                {
+                  "displayname": "Toronto (ca-tor)",
+                  "value": "ca-tor"
+                },
+                {
+                  "displayname": "Washington (us-east)",
+                  "value": "us-east"
+                }
+              ]
+            },
+            {
+              "key": "resource_tags"
+            },
+            {
+              "key": "access_tags"
+            },
+            {
+              "key": "schemas"
+            },
+            {
+              "key": "schema_global_rule"
+            },
+            {
+              "key": "topics"
+            },
+            {
+              "key": "skip_event_streams_s2s_iam_auth_policy"
+            },
+            {
+              "key": "cbr_rules"
+            },
+            {
+              "key": "service_credential_names"
+            },
+            {
+              "key": "quotas"
+            },
+            {
+              "key": "metrics"
+            },
+            {
+              "key": "mirroring_topic_patterns"
+            },
+            {
+              "key": "mirroring"
+            },
+            {
+              "key": "event_streams_key_name"
+            },
+            {
+              "key": "event_streams_key_ring_name"
+            },
+            {
+              "key": "existing_kms_instance_crn"
+            },
+            {
+              "key": "ibmcloud_kms_api_key"
+            },
+            {
+              "key": "kms_endpoint_type"
+            },
+            {
+              "key": "skip_event_streams_kms_auth_policy"
+            },
+            {
+              "key": "existing_kms_key_crn"
+            }
+          ],
+          "iam_permissions": [
+            {
+              "role_crns": [
+                "crn:v1:bluemix:public:iam::::serviceRole:Manager"
+              ],
+              "service_name": "messagehub"
+            },
+            {
+              "role_crns": [
+                "crn:v1:bluemix:public:iam::::role:Editor"
+              ],
+              "service_name": "messagehub"
+            }
+          ],
+          "architecture": {
+            "descriptions": "This architecture creates a Financial Services compliant instance of IBM Event Streams for IBM Cloud in enterprise plan. It also supports the creation of topics and schemas in the Event Streams instance.",
+            "features": [
+              {
+                "title": "Creates an Event Streams instance",
+                "description": "Creates and configures the IBM Cloud Event Streams instance."
+              },
+              {
+                "title": "Creates topics in the Event Streams instance",
+                "description": "Creates topics in IBM Cloud Event Streams instance."
+              },
+              {
+                "title": "Creates schemas in the Event Streams instance",
+                "description": "Creates schemas in IBM Cloud Event Streams instance."
+              },
+              {
+                "title": "Supports mirroring",
+                "description": "Supports mirroring which enables messages in one Event Streams service instance to be continuously copied to a second instance."
+              },
+              {
+                "title": "Suports quotas.",
+                "description": "Supports quotas to control the resources, such as network bandwidth, that a service can consume.."
+              },
+              {
+                "title": "CBR",
+                "description": "Create context-based restriction (CBR) rules for Event Streams instance."
+              },
+              {
+                "title": "Encrypt messages with key management services",
+                "description": "Supports key management encryption (BYOK and KYOK)."
+              },
+              {
+                "title": "Managing users and roles",
+                "description": "Manage IAM service credentials for Event Streams."
+              }
+            ],
+            "diagrams": [
+              {
+                "diagram": {
+                  "caption": "Financial Services compliant instance of IBM Event Streams for IBM Cloud.",
+                  "url": "https://raw.githubusercontent.com/terraform-ibm-modules/terraform-ibm-event-streams/main/reference-architecture/da-enterprise.svg",
+                  "type": "image/svg+xml"
+                },
+                "description": "This architecture creates a Financial Services compliant instance of IBM Event Streams for IBM Cloud."
+              }
+            ]
+          }
         }
       ]
     }