Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCP PostgreSQL onboarding module #25

Merged
merged 32 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
ba84648
add gcp mysql helper module
mattJsonar Nov 6, 2024
b94155e
add gcp sql instance module
mattJsonar Nov 7, 2024
7016e51
add gcp mysql onboarding module
mattJsonar Nov 7, 2024
9481a33
fix typo in documentation URL
mattJsonar Nov 7, 2024
2a4fdf2
add deletion protection
mattJsonar Nov 7, 2024
e4482b8
typos
mattJsonar Nov 8, 2024
5eb29d8
add gcp mysql examples
mattJsonar Nov 12, 2024
70751b0
cleanup examples
mattJsonar Nov 12, 2024
5a41555
set required provider version
mattJsonar Nov 12, 2024
a594819
add gcp mysql module
mattJsonar Nov 12, 2024
d049feb
Apply automatic changes
mattJsonar Nov 12, 2024
e50568a
terraform-docs: automated action
github-actions[bot] Nov 12, 2024
c1cdd06
remove spaces from filter name
mattJsonar Nov 12, 2024
9f44885
Merge branch 'matt/SR-1890-gcp-mysql' of github.com:imperva/terraform…
mattJsonar Nov 12, 2024
4fac2e5
refactor examples
mattJsonar Nov 14, 2024
1fd6dcd
update audit_type description
mattJsonar Nov 15, 2024
c1df39a
add gcp postgresql asset helper module
mattJsonar Nov 15, 2024
90f2dab
add gcp postgres onboarding module
mattJsonar Nov 15, 2024
18a9642
add outputs
mattJsonar Nov 15, 2024
7976eef
add gcp postgres onboarding examples
mattJsonar Nov 18, 2024
e16ee11
add gcp postgresql module
mattJsonar Nov 18, 2024
aa1c1b3
add gcp mysql onboarding module
mattJsonar Nov 7, 2024
40d47b2
add gcp mysql examples
mattJsonar Nov 12, 2024
8d7052f
add gcp postgresql asset helper module
mattJsonar Nov 15, 2024
7c6acdc
add gcp postgres onboarding module
mattJsonar Nov 15, 2024
a65b661
add outputs
mattJsonar Nov 15, 2024
53de1c5
add gcp postgres onboarding examples
mattJsonar Nov 18, 2024
78f5d1b
add gcp postgresql module
mattJsonar Nov 18, 2024
7784d78
Merge branch 'matt/SR-1891-gcp-postgres' of github.com:imperva/terraf…
mattJsonar Nov 18, 2024
2d3743b
fix changelog
mattJsonar Nov 18, 2024
ff6f5f3
Apply automatic changes
mattJsonar Nov 18, 2024
c527041
terraform-docs: automated action
github-actions[bot] Nov 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
### Features
- Aurora PostgreSQL CloudWatch with slow query auditing example
- Google Cloud SQL for MySQL module
- Google Cloud SQL for PostgreSQL module

### Bug Fixes
- Modified Server Host Name of AWS RDS MS SQL SERVER Dsfhub assets
Expand Down
4 changes: 4 additions & 0 deletions DSF_VERSION_COMPATABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,5 +107,9 @@ The following table lists the DSF versions that each module is tested and mainta
<td>onboard-gcp-mysql</td>
<td>4.17+</td>
</tr>
<tr>
<td>onboard-gcp-postgresql</td>
<td>4.17+</td>
</tr>

</table>
2 changes: 1 addition & 1 deletion examples/onboard-gcp-mysql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,4 +45,4 @@ No inputs.
## Outputs

No outputs.
<!-- END_TF_DOCS -->
<!-- END_TF_DOCS -->
60 changes: 60 additions & 0 deletions examples/onboard-gcp-postgresql/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Onboard Google Cloud SQL for PostgreSQL example
This example includes additional prerequisites that will need to be completed to fully utilize the module. More details can be found in the [onboarding documentation](https://docs.imperva.com/bundle/onboarding-databases-to-sonar-reference-guide/page/Cloud-SQL-for-PostgreSQL-Onboarding-Steps_48367600.html).

This example creates 'dsfhub' and 'google' resources. More information regarding authentication to each can be found in the relevant provider documentation:
- [dsfhub](https://registry.terraform.io/providers/imperva/dsfhub/latest/docs)
- [google](https://registry.terraform.io/providers/hashicorp/google/latest/docs)

## Prerequisites
### Service Account
A Google Service Account will need to be created with permissions to read from PubSub subscriptions. This can be done via the ``google-service-account-dsf`` module. Depending on the authentication mechanism chosen, the service account will either need to be attached to a GCP Compute Engine instance or the service account's credentials file will need to be copied to your Agentless Gateway.

### Google PubSub Subscription
A Google logging sink, PubSub topic, and PubSub subscription in addition to a GCP PUBSUB asset in DSF will need to be created in advance. This prerequisite is handled by the ``onboard-gcp-pubsub`` module.

### Database Configuration
Part of the onboarding process involves connecting to your Google PostgreSQL instance and running SQL commands to create an extension. This module includes an example for how to connect to the instance from your local machine and create this.

<!-- BEGIN_TF_DOCS -->
## Requirements

No requirements.

## Providers

| Name | Version |
|------|---------|
| <a name="provider_google"></a> [google](#provider\_google) | n/a |
| <a name="provider_terraform"></a> [terraform](#provider\_terraform) | n/a |

## Modules

| Name | Source | Version |
|------|--------|---------|
| <a name="module_gcp-postgresql-1"></a> [gcp-postgresql-1](#module\_gcp-postgresql-1) | ../../modules/onboard-gcp-postgresql | n/a |
| <a name="module_gcp-postgresql-2"></a> [gcp-postgresql-2](#module\_gcp-postgresql-2) | ../../modules/onboard-gcp-postgresql | n/a |
| <a name="module_gcp-postgresql-3"></a> [gcp-postgresql-3](#module\_gcp-postgresql-3) | ../../modules/onboard-gcp-postgresql | n/a |
| <a name="module_gcp-pubsub-1"></a> [gcp-pubsub-1](#module\_gcp-pubsub-1) | ../../modules/onboard-gcp-pubsub | n/a |
| <a name="module_gcp-pubsub-2"></a> [gcp-pubsub-2](#module\_gcp-pubsub-2) | ../../modules/onboard-gcp-pubsub | n/a |
| <a name="module_gcp-pubsub-3"></a> [gcp-pubsub-3](#module\_gcp-pubsub-3) | ../../modules/onboard-gcp-pubsub | n/a |
| <a name="module_service-account"></a> [service-account](#module\_service-account) | ../../modules/google-service-account-dsf | n/a |

## Resources

| Name | Type |
|------|------|
| [google_sql_user.gcp-postgresql-admin-user-1](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_user) | resource |
| [google_sql_user.gcp-postgresql-admin-user-2](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_user) | resource |
| [google_sql_user.gcp-postgresql-admin-user-3](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_user) | resource |
| [terraform_data.configure_database_1](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
| [terraform_data.configure_database_2](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
| [terraform_data.configure_database_3](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |

## Inputs

No inputs.

## Outputs

No outputs.
<!-- END_TF_DOCS -->
38 changes: 38 additions & 0 deletions examples/onboard-gcp-postgresql/configure_database.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/bin/bash
# Configures Google Postgresql database for auditing by connecting to the
# database using 'psql'
#
# Connection to the database uses PG environment variables
# See postgres documentation for more information:
# https://www.postgresql.org/docs/current/libpq-envars.html
################################################################################

# Settings
current_directory=$(dirname "$(realpath "${BASH_SOURCE[0]}")")
sql_file="${current_directory}/configure_database.sql"

# Functions
function is_pkg_installed {
local pkg="$1"
if ! command -v "${pkg}" &> /dev/null
then
echo "Package '${pkg}' is not installed."
echo "Install on MacOS: brew install libpq"
echo "Install on Ubuntu: apt-get install -y libpq-dev"
echo "Install on CentOS: yum install -y libpq"
echo "Exiting..."
exit 1
else
return 0
fi
}

################################################################################
is_pkg_installed "psql"
if [ ! -r "${sql_file}" ]; then
echo "Unable to read ${sql_file}"
echo "Exiting..."
exit 1
else
psql --file="${sql_file}"
fi
11 changes: 11 additions & 0 deletions examples/onboard-gcp-postgresql/configure_database.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
DO
$$
BEGIN
RAISE NOTICE 'Creating audit extension "pgaudit".';
IF EXISTS (SELECT FROM pg_catalog.pg_extension WHERE extname = 'pgaudit') THEN
RAISE NOTICE 'Audit extension "pgaudit" already exists. Skipping.';
ELSE
CREATE EXTENSION pgaudit;
END IF;
END
$$;
Loading