Azure CosmosDB SQL & MongoDB

CHANGELOG.md

@@ -1,8 +1,10 @@
 # Changelog
 
-## 1.0.6 (TBD)
+## 1.0.6 (2024-09-17)
 
 ### Features
+- Azure CosmosDB API for MongoDB module
+- Azure CosmosDB SQL API module
 - Azure SQL Server module
 
 ## 1.0.5 (2024-08-29)

DSF_VERSION_COMPATABILITY.md

@@ -79,6 +79,14 @@ The following table lists the DSF versions that each module is tested and mainta
       <td>onboard-aws-rds-redshift</td>
       <td>4.17+</td>
    </tr>
+   <tr>
+      <td>onboard-azure-cosmosdb-mongo</td>
+      <td>4.17+</td>
+   </tr>
+   <tr>
+      <td>onboard-azure-cosmosdb-sql</td>
+      <td>4.17+</td>
+   </tr>
    <tr>
       <td>onboard-azure-ms-sql-server</td>
       <td>4.17+</td>

examples/onboard-azure-cosmosdb-mongo/README.md

@@ -0,0 +1,48 @@
+# Onboard Azure Cosmos DB API for MongoDB example
+This example includes additional prerequisites that will need to be completed to fully utilize the module. More details can be found in the [onboarding documentation](https://docs.imperva.com/bundle/onboarding-databases-to-sonar-reference-guide/page/Azure-Cosmos-DB-API-for-MongoDB-Onboarding-Steps_48367240.html).
+
+This example creates 'azurerm' and 'dsfhub' resources. More information regarding authentication to each can be found in the relevant provider documentation:
+- [azurerm](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs)
+- [dsfhub](https://registry.terraform.io/providers/imperva/dsfhub/latest/docs)
+
+## Prerequisites
+This module expects an Event Hub and a Storage Account Container to have been created in advance, in addition to a corresponding existing AZURE EVENTHUB asset in DSF. Both of these prerequisites and all related resources are handled in the ``onboard-azure-eventhub`` module.
+
+### Azure Event Hub Namespace and Event Hub
+Cosmos DB audit logs are sent to an Azure Event Hub and are retrieved by DSF. The Event Hubs are created within an Event Hub Namespace, which can contain one or more Event Hubs. Audit logs of multiple Cosmos DB instances can be sent to a single Event Hub. 
+
+### Azure Storage Account and Container
+Storage Containers are used to store transactional data for the Event Hub import processes, and one Storage Container is required for each Event Hub. These Storage Containers exist within a Storage Account, which may contain multiple Storage Containers.
+
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+No requirements.
+
+## Providers
+
+No providers.
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_azure-cosmosdb-mongo-1"></a> [azure-cosmosdb-mongo-1](#module\_azure-cosmosdb-mongo-1) | ../../modules/onboard-azure-cosmosdb-mongo | n/a |
+| <a name="module_azure-cosmosdb-mongo-2"></a> [azure-cosmosdb-mongo-2](#module\_azure-cosmosdb-mongo-2) | ../../modules/onboard-azure-cosmosdb-mongo | n/a |
+| <a name="module_onboard-cosmos-mongo-eventhub-1"></a> [onboard-cosmos-mongo-eventhub-1](#module\_onboard-cosmos-mongo-eventhub-1) | ../../modules/onboard-azure-eventhub | n/a |
+
+## Resources
+
+No resources.
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_dsfhub_host"></a> [dsfhub\_host](#input\_dsfhub\_host) | n/a | `any` | n/a | yes |
+| <a name="input_dsfhub_token"></a> [dsfhub\_token](#input\_dsfhub\_token) | n/a | `any` | n/a | yes |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->

examples/onboard-azure-cosmosdb-mongo/main.tf

@@ -0,0 +1,139 @@
+locals {
+  azure_eventhub_name       = "cosmosmongoeventhub"
+  azure_location            = "East US"
+  azure_resource_group_name = "My_Resource_Group"
+  azure_subscription_id     = "123456790-wxyz-g8h9-e5f6-a1b2c3d4"
+
+  admin_email = "test@example.com"
+  gateway_id  = "a1b2c3d4-e5f6-g8h9-wxyz-123456790"
+}
+
+################################################################################
+# Providers
+################################################################################
+terraform {
+  required_providers {
+    dsfhub = {
+      source = "imperva/dsfhub"
+    }
+  }
+}
+
+provider "azurerm" {
+  features {}
+  subscription_id = local.azure_subscription_id
+}
+
+variable "dsfhub_host" {}  # TF_VAR_dsfhub_host env variable
+variable "dsfhub_token" {} # TF_VAR_dsfhub_token env variable
+
+provider "dsfhub" {
+  dsfhub_host  = var.dsfhub_host
+  dsfhub_token = var.dsfhub_token
+}
+
+################################################################################
+# Prerequisites
+# 1. Azure Event Hub Namespace and Event Hub. Includes authorization rules for
+#    reading and writing to the Event Hub.
+# 2. Storage Account and Container
+################################################################################
+module "onboard-cosmos-mongo-eventhub-1" {
+  source = "../../modules/onboard-azure-eventhub"
+
+  azure_eventhub_admin_email = local.admin_email
+  azure_eventhub_format      = "Cosmos_Mongo"
+  azure_eventhub_gateway_id  = local.gateway_id
+
+  eventhub_name                          = "cosmosmongoeventhub"
+  eventhub_namespace_location            = local.azure_location
+  eventhub_namespace_name                = "cosmosmongoeventhubns"
+  eventhub_namespace_resource_group_name = local.azure_resource_group_name
+
+  eventhub_resource_group_name = local.azure_resource_group_name
+
+  storage_account_location            = local.azure_location
+  storage_account_name                = "cosmosmongostorageacc"
+  storage_account_resource_group_name = local.azure_resource_group_name
+  storage_container_name              = "cosmosmongostoragecon"
+}
+
+################################################################################
+# Azure Cosmos DB with Mongo API v4.2
+################################################################################
+module "azure-cosmosdb-mongo-1" {
+  source = "../../modules/onboard-azure-cosmosdb-mongo"
+
+  depends_on = [module.onboard-cosmos-mongo-eventhub-1]
+
+  azure_cosmosdb_mongo_admin_email               = local.admin_email
+  azure_cosmosdb_mongo_audit_pull_enabled        = true
+  azure_cosmosdb_mongo_gateway_id                = local.gateway_id
+  azure_cosmosdb_mongo_logs_destination_asset_id = module.onboard-cosmos-mongo-eventhub-1.azure-eventhub-asset.asset_id
+
+  cosmosdb_account_consistency_policy = [
+    {
+      "consistency_level" : "Session"
+    }
+  ]
+  cosmosdb_account_geo_location = [
+    {
+      "failover_priority" : 0,
+      "location" : "eastus",
+      "zone_redundant" : false
+    }
+  ]
+  cosmosdb_account_location             = local.azure_location
+  cosmosdb_account_mongo_server_version = "4.2"
+  cosmosdb_account_name                 = "example-cosmos-mongo"
+  cosmosdb_account_resource_group_name  = local.azure_resource_group_name
+
+  diagnostic_setting_eventhub_authorization_rule_id = module.onboard-cosmos-mongo-eventhub-1.eventhub-write-authorization.id
+  diagnostic_setting_eventhub_name                  = module.onboard-cosmos-mongo-eventhub-1.eventhub.name
+  diagnostic_setting_name                           = "dsfhubdiagnostic"
+}
+
+################################################################################
+# Azure Cosmos DB with Mongo API v4.2 Many-to-One
+################################################################################
+locals {
+  cosmos_mongo_types = toset([
+    "dev",
+    "prod",
+    "uat"
+  ])
+}
+
+module "azure-cosmosdb-mongo-2" {
+  source = "../../modules/onboard-azure-cosmosdb-mongo"
+
+  depends_on = [module.onboard-cosmos-mongo-eventhub-1]
+
+  for_each = local.cosmos_mongo_types
+
+  azure_cosmosdb_mongo_admin_email               = local.admin_email
+  azure_cosmosdb_mongo_audit_pull_enabled        = true
+  azure_cosmosdb_mongo_gateway_id                = local.gateway_id
+  azure_cosmosdb_mongo_logs_destination_asset_id = module.onboard-cosmos-mongo-eventhub-1.azure-eventhub-asset.asset_id
+
+  cosmosdb_account_consistency_policy = [
+    {
+      "consistency_level" : "Session"
+    }
+  ]
+  cosmosdb_account_geo_location = [
+    {
+      "failover_priority" : 0,
+      "location" : "eastus",
+      "zone_redundant" : false
+    }
+  ]
+  cosmosdb_account_location             = local.azure_location
+  cosmosdb_account_mongo_server_version = "4.2"
+  cosmosdb_account_name                 = "example-cosmos-mongo-${each.key}"
+  cosmosdb_account_resource_group_name  = local.azure_resource_group_name
+
+  diagnostic_setting_eventhub_authorization_rule_id = module.onboard-cosmos-mongo-eventhub-1.eventhub-write-authorization.id
+  diagnostic_setting_eventhub_name                  = module.onboard-cosmos-mongo-eventhub-1.eventhub.name
+  diagnostic_setting_name                           = "dsfhubdiagnostic"
+}

examples/onboard-azure-cosmosdb-sql/README.md

@@ -0,0 +1,48 @@
+# Onboard Azure Cosmos DB SQL API example
+This example includes additional prerequisites that will need to be completed to fully utilize the module. More details can be found in the [onboarding documentation](https://docs.imperva.com/bundle/onboarding-databases-to-sonar-reference-guide/page/Azure-Cosmos-DB-SQL-API-Onboarding-Steps_48367255.html).
+
+This example creates 'azapi', 'azurerm' and 'dsfhub' resources. More information regarding authentication to each can be found in the relevant provider documentation:
+- [azapi](https://registry.terraform.io/providers/Azure/azapi/latest/docs)
+- [azurerm](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs)
+- [dsfhub](https://registry.terraform.io/providers/imperva/dsfhub/latest/docs)
+
+## Prerequisites
+This module expects an Event Hub and a Storage Account Container to have been created in advance, in addition to a corresponding existing AZURE EVENTHUB asset in DSF. Both of these prerequisites and all related resources are handled in the ``onboard-azure-eventhub`` module.
+
+### Azure Event Hub Namespace and Event Hub
+Cosmos DB audit logs are sent to an Azure Event Hub and are retrieved by DSF. The Event Hubs are created within an Event Hub Namespace, which can contain one or more Event Hubs. Audit logs of multiple Cosmos DB instances can be sent to a single Event Hub. 
+
+### Azure Storage Account and Container
+Storage Containers are used to store transactional data for the Event Hub import processes, and one Storage Container is required for each Event Hub. These Storage Containers exist within a Storage Account, which may contain multiple Storage Containers.
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+No requirements.
+
+## Providers
+
+No providers.
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_azure-cosmosdb-sql-1"></a> [azure-cosmosdb-sql-1](#module\_azure-cosmosdb-sql-1) | ../../modules/onboard-azure-cosmosdb-sql | n/a |
+| <a name="module_azure-cosmosdb-sql-2"></a> [azure-cosmosdb-sql-2](#module\_azure-cosmosdb-sql-2) | ../../modules/onboard-azure-cosmosdb-sql | n/a |
+| <a name="module_onboard-cosmos-sql-eventhub-1"></a> [onboard-cosmos-sql-eventhub-1](#module\_onboard-cosmos-sql-eventhub-1) | ../../modules/onboard-azure-eventhub | n/a |
+
+## Resources
+
+No resources.
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_dsfhub_host"></a> [dsfhub\_host](#input\_dsfhub\_host) | n/a | `any` | n/a | yes |
+| <a name="input_dsfhub_token"></a> [dsfhub\_token](#input\_dsfhub\_token) | n/a | `any` | n/a | yes |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->