Terraform Google Cloud SQL Postgres module with K8s secret deploy

Terraform module for provisioning GCP SQL Postgres database. It should also deploy the username and password to K8s as a secret. That could be used in setting up cloudsql proxy pod.

Usage

module "postgresql" {
  source  = "AckeeCZ/postgresql/sql"

  project = "${var.project}"
  region = "${var.region}"
  zone = "${var.zone}"
  namespace = "${var.namespace}"
  cluster_ca_certificate = "${module.gke.cluster_ca_certificate}"
  cluster_user = "${module.gke.cluster_username}"
  cluster_pass = "${module.gke.cluster_password}"
  cluster_endpoint = "${module.gke.endpoint}"
  environment = "production"
  instance_tier = "db-n1-standard-1" # optional, default is db-n1-standard-1
  availability_type = "REGIONAL" # REGIONAL for HA setup, ZONAL for single zone
  vault_secret_path = "secret/devops/generated/${TYPE}/${var.project}/${var.environment}" # ${TYPE} should be set to internal for internal projects, external for external projects
  read_replicas = {
    replica-a : {
      instance_tier = "db-custom-1-3840"
      ipv4_enabled  = false
      zone          = "europe-west3-a"
    },
    replica-b : {
      instance_tier = "db-custom-1-3840"
      ipv4_enabled  = false
      zone          = "europe-west3-b"
    },
  }
}

Read replicas

Read replicas are configured from read_replicas parameter map. Key serve as replica name, it is appended to primary's instance_name local variable.

Every read replica have four parameters:

• instance_tier: Instance type for replica, equivalent of primary's instance_tier parameter.
• ipv4_enabled: Availability of public IP address on replica, equivalent of primary's ipv4_enabled parameter.
• zone: Zone where read replicas is deployed. This is bit different from primary's zone parameter. On primary instance, we define "prefered location"
• authorized_networks: List of maps of strings authorized networks allowed to connect to Cloud SQL Read Replica Instance, example: [{name: the_office, cidr: 1.2.3.4/31}] This parameter is optional.

• HA instance will change it's location in case of failover, but read replicas have zone "hard set".

Before you do anything in this module

Install pre-commit hooks by running following commands:

brew install pre-commit terraform-docs
pre-commit install

Dependencies

GKE module: https://gitlab.ack.ee/Infra/terraform-gke-vpc

Example SQL proxy specification

proxy.yaml in infrastuktura-template repo

Requirements

Name	Version
terraform	>= 0.15

Providers

Name	Version
google	n/a
kubernetes	n/a
random	n/a
vault	n/a

Modules

No modules.

Resources

Name	Type
google_compute_global_address.psql_private_ip_address	resource
google_project_iam_member.sqlproxy_role	resource
google_project_service.enable-servicenetworking-api	resource
google_project_service.enable_sqladmin_api	resource
google_service_account.sqlproxy	resource
google_service_account_key.sqlproxy	resource
google_service_networking_connection.private_vpc_connection	resource
google_sql_database.default	resource
google_sql_database_instance.default	resource
google_sql_database_instance.read_replica	resource
google_sql_user.default	resource
google_sql_user.postgres	resource
kubernetes_endpoints.cloudsql	resource
kubernetes_secret.sqlproxy	resource
kubernetes_service.cloudsql	resource
random_id.instance_name_suffix	resource
random_password.postgres_default	resource
random_password.postgres_postgres	resource
vault_generic_secret.default	resource
google_compute_network.default	data source

Inputs

Name	Description	Type	Default	Required
authorized_networks	List of maps of strings authorized networks allowed to connect to Cloud SQL instance, example: [{name: the_office, cidr: 1.2.3.4/31}]	list(map(string))	[]	no
availability_type	The availability type of the Cloud SQL instance, high availability (REGIONAL) or single zone (ZONAL)	string	"ZONAL"	no
backup_location	Location of backups	string	"eu"	no
backup_start_time	The time, when backup starts	string	"03:00"	no
cloudsql_port	CloudSQL's port	number	5432	no
cluster_ca_certificate	Public CA certificate that is the root of trust for the GKE K8s cluster	string	null	no
cluster_endpoint	Cluster control plane endpoint	string	"example.com:8080"	no
cluster_token	Cluster master token, keep always secret!	string	null	no
database_flags	The optional settings.database_flags list of values, where key is name and value is value from documentation: https://www.terraform.io/docs/providers/google/r/sql_database_instance.html	map(string)	{}	no
db_version	Database version	string	"POSTGRES_11"	no
default_dbname_override	Overrides default postgres database name	string	null	no
deletion_protection	Whether or not to allow Terraform to destroy the instance. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply command that deletes the instance will fail.	bool	true	no
disk_autoresize	Configuration to increase storage size automatically	bool	true	no
disk_autoresize_limit	Limit of disk space for autoresize	number	0	no
enable_query_insights	Enable query insights https://cloud.google.com/sql/docs/postgres/insights-overview	bool	true	no
environment	Project enviroment, e.g. stage, production and development	string	"development"	no
instance_tier	The machine type to use	string	"db-custom-1-3840"	no
kubernetes_service_name	Name of kubernetes service	string	"cloudsql"	no
maintenance_window_day	The day, when maintenance window will be performed	string	"7"	no
maintenance_window_hour	The hour, when maintenance window begins	string	"4"	no
name_override	Sets complete CloudSQL instance name	string	null	no
namespace	K8s namespace to where insert Cloud SQL credentials secrets	string	"production"	no
network	GCE VPC used for possible private IP addresses	string	"default"	no
password_length	Password length of postgres users	number	16	no
password_special	Use special characters for passwords of postgres users	bool	true	no
point_in_time_recovery	Enable Point-in-time recovery (effectively turns on WAL)	bool	false	no
private_ip	If set to true, private IP address will get allocated and connect it to VPC network set in var.network in the project -- once enabled, this can't be turned off.	bool	false	no
project	GCP project name	string	n/a	yes
provision_kubernetes_resources	Should we provision anything that needs Kubernetes?	bool	true	no
public_ip	If set to true, public IP address will get allocated	bool	false	no
query_string_length_insights	Insights maximum query length stored in bytes. Between 256 and 4500. Default to 1024.	number	1024	no
random_id_length	Byte length of random ID, used as suffix in SQL name	number	4	no
read_replicas	Map of maps containing name as a key of read_replicas mapa and settings some parameters of read replica. For more information see README part Read replica	map	{}	no
region	GCP region	string	"europe-west3"	no
secondary_zone	secondary GCP region preference	string	null	no
sqlproxy_dependencies	If set to true, we will create dependencies for running SQLproxy - GCP IAM SA, Kubernetes secret and Kubernetes Service	bool	true	no
sqlproxy_service_account_name	SQL instance service account name	string	null	no
transaction_log_retention_days	The number of days of transaction logs we retain for point in time restore, from 1-7.	number	null	no
user_labels	Labels to the instance	map(string)	{}	no
user_suffix	Suffix - used, for instance, when you create a clone. Should include starting dash	string	""	no
vault_secret_path	Path to secret in local vault, used mainly to save gke credentials	string	n/a	yes
zone	The preferred compute engine zone	string	"europe-west3-c"	no

Outputs

Name	Description
instance_connection_name	PSQL instance connection name
instance_name	PSQL instance name
postgres_default_password	PSQL password to default user
postgres_instance_connection_name	PSQL instance connection name
postgres_instance_ip_settings	PSQL instance IP address settings
postgres_instance_name	PSQL instance name
postgres_postgres_password	PSQL password to postgres user
postgres_reader_instance_ip_settings	PSQL instance IP address settings of read replicas