Azure SQL Managed Instance module

CHANGELOG.md

@@ -4,6 +4,7 @@
 
 ### Features
 - Azure Database for PostgreSQL Flexible module
+- Azure SQL Managed Instance module
 
 ## 1.0.6 (2024-09-17)
 

DSF_VERSION_COMPATABILITY.md

@@ -95,5 +95,9 @@ The following table lists the DSF versions that each module is tested and mainta
       <td>onboard-azure-postgresql-flexible</td>
       <td>4.17+</td>
    </tr>
+   <tr>
+      <td>onboard-azure-sql-managed-instance</td>
+      <td>4.17+</td>
+   </tr>
 
 </table>

examples/onboard-azure-sql-managed-instance/README.md

@@ -0,0 +1,58 @@
+# Onboard Azure SQL Managed Instance example
+This example includes additional prerequisites that will need to be completed to fully utilize the module. More details can be found in the [onboarding documentation](https://docs.imperva.com/bundle/onboarding-databases-to-sonar-reference-guide/page/Azure-SQL-Managed-Instance-Onboarding-Steps_48367362.html).
+
+This example creates 'azurerm' and 'dsfhub' resources. More information regarding authentication to each can be found in the relevant provider documentation:
+- [azurerm](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs)
+- [dsfhub](https://registry.terraform.io/providers/imperva/dsfhub/latest/docs)
+
+## Prerequisites
+This module expects an Event Hub and a Storage Account Container to have been created in advance, in addition to a corresponding existing AZURE EVENTHUB asset in DSF. Both of these prerequisites and all related resources are handled in the ``onboard-azure-eventhub`` module. It also expects a method to connect to the SQL Managed instance to create a server audit policy.
+
+### Azure Event Hub Namespace and Event Hub
+SQL Managed instance audit logs are sent to an Azure Event Hub and are retrieved by DSF. The Event Hubs are created within an Event Hub Namespace, which can contain one or more Event Hubs. Audit logs of multiple SQL Managed instances can be sent to a single Event Hub. 
+
+### Azure Storage Account and Container
+Storage Containers are used to store transactional data for the Event Hub import processes, and one Storage Container is required for each Event Hub. These Storage Containers exist within a Storage Account, which may contain multiple Storage Containers.
+
+### Database Configuration
+Part of the onboarding process involves connecting to your SQL Managed instance and running SQL commands to create an audit policy. This module includes an example for how to connect to the instance from your local machine and create it. 
+
+**Note:** This example requires the ``sqlcmd`` client to be installed, as well as for the newly created SQL Managed instance to be accessible from your local machine. 
+
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+No requirements.
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_http"></a> [http](#provider\_http) | n/a |
+| <a name="provider_terraform"></a> [terraform](#provider\_terraform) | n/a |
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_onboard-azure-sql-managed-instance-eventhub-1"></a> [onboard-azure-sql-managed-instance-eventhub-1](#module\_onboard-azure-sql-managed-instance-eventhub-1) | ../../modules/onboard-azure-eventhub | n/a |
+| <a name="module_sql-managed-instance-1"></a> [sql-managed-instance-1](#module\_sql-managed-instance-1) | ../../modules/onboard-azure-sql-managed-instance | n/a |
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [terraform_data.configure_database-1](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
+| [http_http.my-ip](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_dsfhub_host"></a> [dsfhub\_host](#input\_dsfhub\_host) | n/a | `any` | n/a | yes |
+| <a name="input_dsfhub_token"></a> [dsfhub\_token](#input\_dsfhub\_token) | n/a | `any` | n/a | yes |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->

examples/onboard-azure-sql-managed-instance/configure_audit_policy.sql

@@ -0,0 +1,101 @@
+DECLARE  
+  @server_audit AS VARCHAR(50) = '$(server_audit)',
+  @server_audit_spec_name AS VARCHAR(50) = '$(audit_spec_name)',
+  @server_audit_spec AS VARCHAR(3000),
+  @server_audit_status INT,
+  @sql_command AS VARCHAR(3000)
+
+USE master;
+
+-- Create server audit
+IF (EXISTS (SELECT * FROM sys.dm_server_audit_status where name = @server_audit))
+  BEGIN 
+    PRINT 'Server audit "' + @server_audit + '" already exists.';
+  END
+ELSE 
+  BEGIN 
+    PRINT 'Creating server audit "' + @server_audit + '"';
+
+    SET @sql_command = 'CREATE SERVER AUDIT ' + @server_audit + ' TO EXTERNAL_MONITOR';
+    EXECUTE(@sql_command);
+  END;
+
+-- Enable server audit
+select @server_audit_status = status FROM sys.dm_server_audit_status where name = @server_audit;
+
+IF (@server_audit_status = 1)
+  BEGIN
+    PRINT 'Server audit "' + @server_audit + '" is already enabled.';
+  END;
+ELSE 
+  BEGIN 
+    PRINT 'Enabling server audit "' + @server_audit + '"';
+
+    SET @sql_command = 'ALTER SERVER AUDIT ' + @server_audit + ' WITH (STATE = ON)';
+    EXECUTE(@sql_command);
+  END;
+
+
+-- Create server audit specification
+IF (EXISTS (SELECT * FROM sys.server_audit_specifications where name = @server_audit_spec_name))
+  BEGIN
+    PRINT 'Server audit specification "' + @server_audit_spec_name + '" already exists.';
+  END;
+ELSE 
+  BEGIN 
+    PRINT 'Creating server audit specification "' + @server_audit_spec_name + '"';
+
+    -- This creates a server audit specification that captures all server-level and database-level events.
+    -- Modify or add additional groups as needed.
+    -- For all action groups available, see https://learn.microsoft.com/en-us/sql/relational-databases/security/auditing/sql-server-audit-action-groups-and-actions?view=sql-server-ver15
+    SET @server_audit_spec = 'ADD (APPLICATION_ROLE_CHANGE_PASSWORD_GROUP),
+ADD (AUDIT_CHANGE_GROUP),
+ADD (BACKUP_RESTORE_GROUP),
+ADD (BATCH_COMPLETED_GROUP),
+ADD (BATCH_STARTED_GROUP),
+ADD (BROKER_LOGIN_GROUP),
+ADD (DATABASE_CHANGE_GROUP),
+ADD (DATABASE_LOGOUT_GROUP),
+ADD (DATABASE_MIRRORING_LOGIN_GROUP),
+ADD (DATABASE_OBJECT_ACCESS_GROUP),
+ADD (DATABASE_OBJECT_CHANGE_GROUP),
+ADD (DATABASE_OBJECT_OWNERSHIP_CHANGE_GROUP),
+ADD (DATABASE_OBJECT_PERMISSION_CHANGE_GROUP),
+ADD (DATABASE_OPERATION_GROUP),
+ADD (DATABASE_OWNERSHIP_CHANGE_GROUP),
+ADD (DATABASE_PERMISSION_CHANGE_GROUP),
+ADD (DATABASE_PRINCIPAL_CHANGE_GROUP),
+ADD (DATABASE_PRINCIPAL_IMPERSONATION_GROUP),
+ADD (DATABASE_ROLE_MEMBER_CHANGE_GROUP),
+ADD (DBCC_GROUP),
+ADD (FAILED_DATABASE_AUTHENTICATION_GROUP),
+ADD (FAILED_LOGIN_GROUP),
+ADD (FULLTEXT_GROUP),
+ADD (LOGIN_CHANGE_PASSWORD_GROUP),
+ADD (LOGOUT_GROUP),
+ADD (SCHEMA_OBJECT_ACCESS_GROUP),
+ADD (SCHEMA_OBJECT_CHANGE_GROUP),
+ADD (SCHEMA_OBJECT_OWNERSHIP_CHANGE_GROUP),
+ADD (SCHEMA_OBJECT_PERMISSION_CHANGE_GROUP),
+ADD (SERVER_OBJECT_CHANGE_GROUP),
+ADD (SERVER_OBJECT_OWNERSHIP_CHANGE_GROUP),
+ADD (SERVER_OBJECT_PERMISSION_CHANGE_GROUP),
+ADD (SERVER_OPERATION_GROUP),
+ADD (SERVER_PERMISSION_CHANGE_GROUP),
+ADD (SERVER_PRINCIPAL_CHANGE_GROUP),
+ADD (SERVER_PRINCIPAL_IMPERSONATION_GROUP),
+ADD (SERVER_ROLE_MEMBER_CHANGE_GROUP),
+ADD (SERVER_STATE_CHANGE_GROUP),
+ADD (SUCCESSFUL_DATABASE_AUTHENTICATION_GROUP),
+ADD (SUCCESSFUL_LOGIN_GROUP),
+ADD (TRACE_CHANGE_GROUP),
+ADD (TRANSACTION_GROUP),
+ADD (USER_CHANGE_PASSWORD_GROUP),
+ADD (USER_DEFINED_AUDIT_GROUP)
+WITH (STATE = ON)';
+
+    SET @sql_command = 'CREATE SERVER AUDIT SPECIFICATION ' + @server_audit_spec_name + ' FOR SERVER AUDIT ' + @server_audit + ' ' + @server_audit_spec;
+    EXECUTE(@sql_command);
+  END;
+
+GO

examples/onboard-azure-sql-managed-instance/configure_database.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+# Creates an audit policy on an SQL Server instance using the 'sqlcmd' client
+
+# Settings
+current_directory=$(dirname "$(realpath "${BASH_SOURCE[0]}")")
+policy_sql_file="${current_directory}/configure_audit_policy.sql"
+
+# Functions
+function is_pkg_installed {
+  local pkg="$1"
+  if ! command -v "${pkg}" &> /dev/null
+  then
+    echo "Package '${pkg}' is not installed."
+    echo "Please see https://learn.microsoft.com/en-us/sql/tools/sqlcmd/sqlcmd-utility for installation instructions for your OS."
+    echo "Exiting..."
+    exit 1
+  else 
+    return 0
+  fi 
+}
+
+is_pkg_installed "sqlcmd"
+
+# Create server audit with server audit specification
+if [ ! -r "${policy_sql_file}" ]; then 
+  echo "Unable to read ${policy_sql_file}"
+  echo "Exiting..."
+  exit 1
+else 
+  sqlcmd -S tcp:${ENDPOINT} -U ${ADMIN_USER} -P ${ADMIN_PASSWORD} -v server_audit=${SERVER_AUDIT_NAME} -v audit_spec_name=${SERVER_AUDIT_SPEC_NAME} -C < ${policy_sql_file}
+fi

examples/onboard-azure-sql-managed-instance/main.tf

@@ -0,0 +1,146 @@
+locals {
+  azure_location            = "East US"
+  azure_resource_group_name = "My_Resource_Group"
+  azure_subscription_id     = "123456790-wxyz-g8h9-e5f6-a1b2c3d4"
+
+  managed_instance_admin          = "testadmin"
+  managed_instance_admin_password = "Abcd1234"
+  server_audit_name               = "tfmanagedsqlserveraudit"
+  server_audit_spec_name          = "tfmanagedsqlserverauditspec"
+
+  admin_email = "test@example.com"
+  gateway_id  = "a1b2c3d4-e5f6-g8h9-wxyz-123456790"
+}
+
+################################################################################
+# Providers
+################################################################################
+terraform {
+  required_providers {
+    dsfhub = {
+      source = "imperva/dsfhub"
+    }
+  }
+}
+
+provider "azurerm" {
+  features {}
+  subscription_id = local.azure_subscription_id
+}
+
+variable "dsfhub_host" {}  # TF_VAR_dsfhub_host env variable
+variable "dsfhub_token" {} # TF_VAR_dsfhub_token env variable
+
+provider "dsfhub" {
+  dsfhub_host  = var.dsfhub_host
+  dsfhub_token = var.dsfhub_token
+}
+
+################################################################################
+# Prerequisites
+# 1. Azure Event Hub Namespace and Event Hub. Includes authorization rules for
+#    reading and writing to the Event Hub.
+# 2. Storage Account and Container
+# 3. Method to create server audit policy on the instance.
+################################################################################
+# 1 and 2
+module "onboard-azure-sql-managed-instance-eventhub-1" {
+  source = "../../modules/onboard-azure-eventhub"
+
+  azure_eventhub_admin_email = local.admin_email
+  azure_eventhub_format      = "AzureSQL_Managed"
+  azure_eventhub_gateway_id  = local.gateway_id
+
+  eventhub_name                          = "sqlmanagedeventhub"
+  eventhub_namespace_location            = local.azure_location
+  eventhub_namespace_name                = "sqlmanagedeventhubns"
+  eventhub_namespace_resource_group_name = local.azure_resource_group_name
+
+  eventhub_resource_group_name = local.azure_resource_group_name
+
+  storage_account_location            = local.azure_location
+  storage_account_name                = "sqlmanagedstorageacc"
+  storage_account_resource_group_name = local.azure_resource_group_name
+  storage_container_name              = "sqlmanagedstoragecon"
+}
+
+# 3. Run shell script locally to create the server audit policy on the newly created instance
+# Get current IP address
+data "http" "my-ip" {
+  url = "http://icanhazip.com"
+}
+
+# Create security group rule for current IP
+locals {
+  allow_tf_localhost_security_rule = {
+    name                       = "allow_tf_localhost_security_rule"
+    access                     = "Allow"
+    description                = "allow terraform host to connect to managed instance"
+    destination_address_prefix = "*"
+    destination_port_range     = "3342"
+    direction                  = "Inbound"
+    priority                   = 1001
+    protocol                   = "Tcp"
+    source_address_prefix      = chomp(data.http.my-ip.response_body)
+    source_port_range          = "*"
+  }
+}
+
+# Construct public endpoint
+locals {
+  fqdn_parsing                     = regex("([^.]+)(.*)", module.sql-managed-instance-1.azure-ms-sql-managed-instance.fqdn) # Split after instance name
+  managed_instance_public_endpoint = "${local.fqdn_parsing[0]}.public${local.fqdn_parsing[1]},3342"                         # Add ".public" and public port
+}
+
+# Create server audit policy
+resource "terraform_data" "configure_database-1" {
+  depends_on = [module.sql-managed-instance-1]
+
+  triggers_replace = 1
+
+  provisioner "local-exec" {
+    environment = {
+      ADMIN_USER     = local.managed_instance_admin
+      ADMIN_PASSWORD = local.managed_instance_admin_password
+      ENDPOINT       = local.managed_instance_public_endpoint
+
+      SERVER_AUDIT_NAME      = local.server_audit_name
+      SERVER_AUDIT_SPEC_NAME = local.server_audit_spec_name
+    }
+
+    command = "./configure_database.sh"
+
+    on_failure = fail
+  }
+}
+
+################################################################################
+# Azure SQL Managed Instance
+################################################################################
+module "sql-managed-instance-1" {
+  source = "../../modules/onboard-azure-sql-managed-instance"
+
+  azure_sql_managed_instance_admin_email               = local.admin_email
+  azure_sql_managed_instance_audit_pull_enabled        = true
+  azure_sql_managed_instance_gateway_id                = local.gateway_id
+  azure_sql_managed_instance_location                  = local.azure_location
+  azure_sql_managed_instance_logs_destination_asset_id = module.onboard-azure-sql-managed-instance-eventhub-1.azure-eventhub-asset.asset_id
+
+  diagnostic_setting_eventhub_authorization_rule_id = module.onboard-azure-sql-managed-instance-eventhub-1.eventhub-write-authorization.id
+  diagnostic_setting_eventhub_name                  = module.onboard-azure-sql-managed-instance-eventhub-1.eventhub.name
+  diagnostic_setting_name                           = "dsfhubdiagnostic"
+
+  managed_instance_administrator_login          = local.managed_instance_admin
+  managed_instance_administrator_login_password = local.managed_instance_admin_password
+  managed_instance_location                     = local.azure_location
+  managed_instance_name                         = "tf-example-sql-managed-instance"
+  managed_instance_public_data_endpoint_enabled = true
+  managed_instance_resource_group_name          = local.azure_resource_group_name
+
+  route_table_resource_group_name = local.azure_resource_group_name
+
+  security_group_resource_group_name = local.azure_resource_group_name
+  security_group_security_rules      = [local.allow_tf_localhost_security_rule]
+
+  virtual_network_resource_group_name = local.azure_resource_group_name
+}