Add azure postgres flexible module

examples/onboard-azure-postgresql-flexible/README.md

@@ -0,0 +1,61 @@
+# Onboard Azure Database for PostgreSQL Flexible example
+This example includes additional prerequisites that will need to be completed to fully utilize the module. More details can be found in the [onboarding documentation](https://docs.imperva.com/bundle/onboarding-databases-to-sonar-reference-guide/page/Azure-Database-for-PostgreSQL-Flexible-Server-Onboarding-Steps_84345082.html).
+
+This example creates 'azurerm' and 'dsfhub' resources. More information regarding authentication to each can be found in the relevant provider documentation:
+- [azurerm](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs)
+- [dsfhub](https://registry.terraform.io/providers/imperva/dsfhub/latest/docs)
+
+## Prerequisites
+This module expects an Event Hub and a Storage Account Container to have been created in advance, in addition to a corresponding existing AZURE EVENTHUB asset in DSF. Both of these prerequisites and all related resources are handled in the ``onboard-azure-eventhub`` module. It also expects a method to connect to the PostgreSQL instance to create the ``pgaudit`` extension.
+
+### Azure Event Hub Namespace and Event Hub
+PostgreSQL audit logs are sent to an Azure Event Hub and are retrieved by DSF. The Event Hubs are created within an Event Hub Namespace, which can contain one or more Event Hubs. Audit logs of multiple PostgreSQL instances can be sent to a single Event Hub. 
+
+### Azure Storage Account and Container
+Storage Containers are used to store transactional data for the Event Hub import processes, and one Storage Container is required for each Event Hub. These Storage Containers exist within a Storage Account, which may contain multiple Storage Containers.
+
+### Database Configuration
+Part of the onboarding process involves connecting to your Azure PostgreSQL instance and running SQL commands to create a role. This module includes an example for how to connect to the instance from your local machine and create this. 
+
+
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+No requirements.
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_azurerm"></a> [azurerm](#provider\_azurerm) | n/a |
+| <a name="provider_http"></a> [http](#provider\_http) | n/a |
+| <a name="provider_terraform"></a> [terraform](#provider\_terraform) | n/a |
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_azure-postgresql-flexible-1"></a> [azure-postgresql-flexible-1](#module\_azure-postgresql-flexible-1) | ../../modules/onboard-azure-postgresql-flexible | n/a |
+| <a name="module_azure-postgresql-flexible-2"></a> [azure-postgresql-flexible-2](#module\_azure-postgresql-flexible-2) | ../../modules/onboard-azure-postgresql-flexible | n/a |
+| <a name="module_azure-postgresql-flexible-3"></a> [azure-postgresql-flexible-3](#module\_azure-postgresql-flexible-3) | ../../modules/onboard-azure-postgresql-flexible | n/a |
+| <a name="module_onboard-azure-postgresql-flexible-eventhub-1"></a> [onboard-azure-postgresql-flexible-eventhub-1](#module\_onboard-azure-postgresql-flexible-eventhub-1) | ../../modules/onboard-azure-eventhub | n/a |
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [azurerm_postgresql_flexible_server_firewall_rule.my-ip](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/postgresql_flexible_server_firewall_rule) | resource |
+| [terraform_data.configure_database](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
+| [http_http.my-ip](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) | data source |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_dsfhub_host"></a> [dsfhub\_host](#input\_dsfhub\_host) | n/a | `any` | n/a | yes |
+| <a name="input_dsfhub_token"></a> [dsfhub\_token](#input\_dsfhub\_token) | n/a | `any` | n/a | yes |
+
+## Outputs
+
+No outputs.
+<!-- END_TF_DOCS -->

examples/onboard-azure-postgresql-flexible/configure_database.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+# Configures Azure Postgresql Flexible database for auditing by connecting to the 
+# database using 'psql'
+# 
+# Connection to the database uses PG environment variables
+# See postgres documentation for more information: 
+# https://www.postgresql.org/docs/current/libpq-envars.html
+################################################################################
+
+# Settings
+current_directory=$(dirname "$(realpath "${BASH_SOURCE[0]}")")
+sql_file="${current_directory}/configure_database.sql"
+
+# Functions
+function is_pkg_installed {
+  local pkg="$1"
+  if ! command -v "${pkg}" &> /dev/null
+  then
+    echo "Package '${pkg}' is not installed."
+    echo "Install on MacOS:  brew install libpq"
+    echo "Install on Ubuntu: apt-get install -y libpq-dev"
+    echo "Install on CentOS: yum install -y libpq"
+    echo "Exiting..."
+    exit 1
+  else 
+    return 0
+  fi 
+}
+
+################################################################################
+is_pkg_installed "psql"
+if [ ! -r "${sql_file}" ]; then 
+  echo "Unable to read ${sql_file}"
+  echo "Exiting..."
+  exit 1
+else 
+  psql --file="${sql_file}"
+fi

examples/onboard-azure-postgresql-flexible/configure_database.sql

@@ -0,0 +1,11 @@
+DO
+$$
+BEGIN
+  RAISE NOTICE 'Creating audit extension "pgaudit".';
+  IF EXISTS (SELECT FROM pg_catalog.pg_extension WHERE extname = 'pgaudit') THEN
+    RAISE NOTICE 'Audit extension "pgaudit" already exists. Skipping.';
+  ELSE
+    CREATE EXTENSION pgaudit;
+  END IF;
+END
+$$;

examples/onboard-azure-postgresql-flexible/main.tf

@@ -0,0 +1,202 @@
+locals {
+  azure_location            = "East US"
+  azure_resource_group_name = "My_Resource_Group"
+  azure_subscription_id     = "123456790-wxyz-g8h9-e5f6-a1b2c3d4"
+
+  postgresql_admin          = "exampleadmin"
+  postgresql_admin_password = "Abcd1234"
+
+  admin_email = "test@example.com"
+  gateway_id  = "a1b2c3d4-e5f6-g8h9-wxyz-123456790"
+}
+
+################################################################################
+# Providers
+################################################################################
+terraform {
+  required_providers {
+    dsfhub = {
+      source = "imperva/dsfhub"
+    }
+  }
+}
+
+provider "azurerm" {
+  features {}
+  subscription_id = local.azure_subscription_id
+}
+
+variable "dsfhub_host" {}  # TF_VAR_dsfhub_host env variable
+variable "dsfhub_token" {} # TF_VAR_dsfhub_token env variable
+
+provider "dsfhub" {
+  dsfhub_host  = var.dsfhub_host
+  dsfhub_token = var.dsfhub_token
+}
+
+################################################################################
+# Prerequisites
+# 1. Azure Event Hub Namespace and Event Hub. Includes authorization rules for
+#    reading and writing to the Event Hub.
+# 2. Storage Account and Container
+# 3. Method to create 'pgaudit' extension on the postgresql instance.
+################################################################################
+# 1 and 2
+module "onboard-azure-postgresql-flexible-eventhub-1" {
+  source = "../../modules/onboard-azure-eventhub"
+
+  azure_eventhub_admin_email = local.admin_email
+  azure_eventhub_format      = "Postgresql_Flexible"
+  azure_eventhub_gateway_id  = local.gateway_id
+
+  eventhub_name                          = "pgflexeventhub"
+  eventhub_namespace_location            = local.azure_location
+  eventhub_namespace_name                = "pgflexeventhubns"
+  eventhub_namespace_resource_group_name = local.azure_resource_group_name
+
+  eventhub_resource_group_name = local.azure_resource_group_name
+
+  storage_account_location            = local.azure_location
+  storage_account_name                = "pgflexstorageacc"
+  storage_account_resource_group_name = local.azure_resource_group_name
+  storage_container_name              = "pgflexstoragecon"
+}
+
+# 3. Run shell script locally to create extension on the newly created postgresql instance. 
+# Get current IP address
+data "http" "my-ip" {
+  url = "http://icanhazip.com"
+}
+
+# Allow access to postgresql instance from current IP address
+resource "azurerm_postgresql_flexible_server_firewall_rule" "my-ip" {
+  name             = "dsfhub-example"
+  server_id        = module.azure-postgresql-flexible-1.postgresql-server.id
+  start_ip_address = chomp(data.http.my-ip.response_body)
+  end_ip_address   = chomp(data.http.my-ip.response_body)
+}
+
+# Run shell script
+resource "terraform_data" "configure_database" {
+  depends_on = [module.azure-postgresql-flexible-1]
+
+  provisioner "local-exec" {
+
+    environment = {
+      PGHOST     = module.azure-postgresql-flexible-1.postgresql-server.fqdn
+      PGUSER     = local.postgresql_admin
+      PGPASSWORD = local.postgresql_admin_password
+      PGPORT     = "5432"
+      PGDATABASE = "postgres"
+    }
+
+    command = "./configure_database.sh"
+
+    on_failure = fail
+  }
+}
+
+################################################################################
+# Azure PostgreSQL Flexible 16
+################################################################################
+module "azure-postgresql-flexible-1" {
+  source = "../../modules/onboard-azure-postgresql-flexible"
+
+  azure_postgresql_flexible_admin_email               = local.admin_email
+  azure_postgresql_flexible_audit_pull_enabled        = true
+  azure_postgresql_flexible_gateway_id                = local.gateway_id
+  azure_postgresql_flexible_location                  = local.azure_location
+  azure_postgresql_flexible_logs_destination_asset_id = module.onboard-azure-postgresql-flexible-eventhub-1.azure-eventhub-asset.asset_id
+
+  diagnostic_setting_eventhub_authorization_rule_id = module.onboard-azure-postgresql-flexible-eventhub-1.eventhub-write-authorization.id
+  diagnostic_setting_eventhub_name                  = module.onboard-azure-postgresql-flexible-eventhub-1.eventhub.name
+
+  server_administrator_login           = local.postgresql_admin
+  server_administrator_password        = local.postgresql_admin_password
+  server_location                      = local.azure_location
+  server_name                          = "example-azure-pg-flex"
+  server_postgres_version              = 16
+  server_public_network_access_enabled = true
+  server_resource_group_name           = local.azure_resource_group_name
+}
+
+################################################################################
+# Azure PostgreSQL Flexible 16 with Slow Query Monitoring
+################################################################################
+module "azure-postgresql-flexible-2" {
+  source = "../../modules/onboard-azure-postgresql-flexible"
+
+  azure_postgresql_flexible_admin_email               = local.admin_email
+  azure_postgresql_flexible_audit_pull_enabled        = true
+  azure_postgresql_flexible_gateway_id                = local.gateway_id
+  azure_postgresql_flexible_location                  = local.azure_location
+  azure_postgresql_flexible_logs_destination_asset_id = module.onboard-azure-postgresql-flexible-eventhub-1.azure-eventhub-asset.asset_id
+
+  diagnostic_setting_eventhub_authorization_rule_id = module.onboard-azure-postgresql-flexible-eventhub-1.eventhub-write-authorization.id
+  diagnostic_setting_eventhub_name                  = module.onboard-azure-postgresql-flexible-eventhub-1.eventhub.name
+
+  server_administrator_login           = local.postgresql_admin
+  server_administrator_password        = local.postgresql_admin_password
+  server_location                      = local.azure_location
+  server_name                          = "example-azure-pg-flex"
+  server_postgres_version              = 16
+  server_public_network_access_enabled = true
+  server_resource_group_name           = local.azure_resource_group_name
+
+  server_configurations = [
+    {
+      "name" : "pgaudit.log",
+      "value" : "all"
+    },
+    {
+      "name" : "log_line_prefix",
+      "value" : "SONAR_AUDIT=1|TIMESTAMP=%m|APPLICATION_NAME=%a|USER=%u|DATABASE=%d|REMOTE_HOST_AND_PORT=%r|SQL_STATE=%e|SESSION_ID=%c|SESSION_START=%s|PROCESS_ID[%p]|VIRTUAL_TRANSACTION_ID=%v|TRANSACTION_ID=%x|"
+    },
+    {
+      "name" : "azure.extensions",
+      "value" : "pgaudit"
+    },
+    {
+      "name" : "shared_preload_libraries",
+      "value" : "pgaudit"
+    },
+    {
+      "name" : "log_min_duration_statement",
+      "value" : 1000
+    }
+  ]
+}
+
+################################################################################
+# Azure PostgreSQL Flexible 16 Many-to-One
+################################################################################
+locals {
+  postgres_types = toset([
+    "dev",
+    "prod",
+    "uat"
+  ])
+}
+
+module "azure-postgresql-flexible-3" {
+  source = "../../modules/onboard-azure-postgresql-flexible"
+
+  for_each = local.postgres_types
+
+  azure_postgresql_flexible_admin_email               = local.admin_email
+  azure_postgresql_flexible_audit_pull_enabled        = true
+  azure_postgresql_flexible_gateway_id                = local.gateway_id
+  azure_postgresql_flexible_location                  = local.azure_location
+  azure_postgresql_flexible_logs_destination_asset_id = module.onboard-azure-postgresql-flexible-eventhub-1.azure-eventhub-asset.asset_id
+
+  diagnostic_setting_eventhub_authorization_rule_id = module.onboard-azure-postgresql-flexible-eventhub-1.eventhub-write-authorization.id
+  diagnostic_setting_eventhub_name                  = module.onboard-azure-postgresql-flexible-eventhub-1.eventhub.name
+
+  server_administrator_login           = local.postgresql_admin
+  server_administrator_password        = local.postgresql_admin_password
+  server_location                      = local.azure_location
+  server_name                          = "example-azure-pg-flex-${each.key}"
+  server_postgres_version              = 16
+  server_public_network_access_enabled = true
+  server_resource_group_name           = local.azure_resource_group_name
+}

modules/azurerm-postgresql-flexible-server/README.md

@@ -0,0 +1,43 @@
+<!-- BEGIN_TF_DOCS -->
+## Requirements
+
+No requirements.
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_azurerm"></a> [azurerm](#provider\_azurerm) | n/a |
+
+## Modules
+
+No modules.
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [azurerm_postgresql_flexible_server.this](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/postgresql_flexible_server) | resource |
+
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|------|---------|:--------:|
+| <a name="input_administrator_login"></a> [administrator\_login](#input\_administrator\_login) | The Administrator login for the PostgreSQL Flexible Server. | `string` | n/a | yes |
+| <a name="input_administrator_password"></a> [administrator\_password](#input\_administrator\_password) | The Password associated with the administrator\_login for the PostgreSQL Flexible Server. | `string` | n/a | yes |
+| <a name="input_location"></a> [location](#input\_location) | The Azure Region where the PostgreSQL Flexible Server should exist. Changing this forces a new PostgreSQL Flexible Server to be created. | `string` | n/a | yes |
+| <a name="input_name"></a> [name](#input\_name) | The name which should be used for this PostgreSQL Flexible Server. Changing this forces a new PostgreSQL Flexible Server to be created. | `string` | n/a | yes |
+| <a name="input_postgres_version"></a> [postgres\_version](#input\_postgres\_version) | The version of PostgreSQL Flexible Server to use. Possible values are 11, 12, 13, 14, 15 and 16. Defaults to 16. | `number` | `16` | no |
+| <a name="input_public_network_access_enabled"></a> [public\_network\_access\_enabled](#input\_public\_network\_access\_enabled) | Specifies whether this PostgreSQL Flexible Server is publicly accessible. Defaults to true. | `bool` | `true` | no |
+| <a name="input_resource_group_name"></a> [resource\_group\_name](#input\_resource\_group\_name) | The name of the Resource Group where the PostgreSQL Flexible Server should exist. Changing this forces a new PostgreSQL Flexible Server to be created. | `string` | n/a | yes |
+| <a name="input_sku_name"></a> [sku\_name](#input\_sku\_name) | The SKU Name for the PostgreSQL Flexible Server. The name of the SKU, follows the tier + name pattern (e.g. B\_Standard\_B1ms, GP\_Standard\_D2s\_v3, MO\_Standard\_E4s\_v3). Defaults to GP\_Standard\_D2s\_v3. | `string` | `"GP_Standard_D2s_v3"` | no |
+| <a name="input_storage_mb"></a> [storage\_mb](#input\_storage\_mb) | The max storage allowed for the PostgreSQL Flexible Server. Possible values are 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4193280, 4194304, 8388608, 16777216 and 33553408. Defaults to 32768. | `number` | `"32768"` | no |
+| <a name="input_storage_tier"></a> [storage\_tier](#input\_storage\_tier) | The name of storage performance tier for IOPS of the PostgreSQL Flexible Server. Possible values are P4, P6, P10, P15,P20, P30,P40, P50,P60, P70 or P80. Default value is dependant on the storage\_mb value. Please see the storage\_tier defaults based on storage\_mb table [here](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/postgresql_flexible_server#storage_tier-defaults-based-on-storage_mb). | `string` | `null` | no |
+| <a name="input_tags"></a> [tags](#input\_tags) | A mapping of tags to assign to the resource. | `map(string)` | `null` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_this"></a> [this](#output\_this) | Azure PostgreSQL Flexible server. |
+<!-- END_TF_DOCS -->

modules/azurerm-postgresql-flexible-server/main.tf

@@ -0,0 +1,18 @@
+resource "azurerm_postgresql_flexible_server" "this" {
+  lifecycle {
+    ignore_changes = [zone]
+  }
+
+  administrator_login           = var.administrator_login
+  administrator_password        = var.administrator_password
+  create_mode                   = "Default"
+  location                      = var.location
+  name                          = var.name
+  public_network_access_enabled = var.public_network_access_enabled
+  resource_group_name           = var.resource_group_name
+  sku_name                      = var.sku_name
+  storage_mb                    = var.storage_mb
+  storage_tier                  = var.storage_tier
+  tags                          = var.tags
+  version                       = var.postgres_version
+}

modules/azurerm-postgresql-flexible-server/outputs.tf

@@ -0,0 +1,4 @@
+output "this" {
+  description = "Azure PostgreSQL Flexible server."
+  value       = azurerm_postgresql_flexible_server.this
+}