terraform-aws-arc-opensearch

Overview

Terraform module for Amazon OpenSearch provides a flexible and scalable way to deploy and manage OpenSearch clusters, with support for both serverless and managed (provisioned) deployment options.

Usage

See the example/vpc folder for a working module example.

################################################################################
## opensearch
################################################################################
module "opensearch" {
  source                       = "sourcefuse/arc-opensearch/aws"
  version                      = "0.1.2"
  namespace                    = var.namespace
  environment                  = var.environment
  name                         = var.name
  engine_version               = var.engine_version
  instance_type                = var.instance_type
  instance_count               = var.instance_count
  enable_vpc_options           = true
  vpc_id                       = data.aws_vpc.default.id
  subnet_ids                   = local.private_subnet_ids
  ingress_rules                = local.ingress_rules
  egress_rules                 = local.egress_rules

  tags = module.tags.tags
}

################################################################################
## opensearch serverless
################################################################################
 module "opensearch_serverless" {
  source                       = "sourcefuse/arc-opensearch/aws"
  version                      = "0.1.2"
  enable_serverless            = true
  namespace                    = var.namespace
  environment                  = var.environment
  name                         = var.name
  ingress_rules                = local.ingress_rules
  egress_rules                 = local.egress_rules
  subnet_ids                   = local.private_subnet_ids
  vpc_id                       = data.aws_vpc.default.id
  data_lifecycle_policy_rules  = local.data_lifecycle_policy_rules
  access_policy_rules          = local.access_policy_rules
  tags                         = module.tags.tags

}

See the example/public folder if you want your os to be public

################################################################################
## opensearch
################################################################################
module "opensearch" {
  source                         = "sourcefuse/arc-opensearch/aws"
  version                        = "1.0.3"
  namespace                      = var.namespace
  environment                    = var.environment
  name                           = var.name
  engine_version                 = var.engine_version
  instance_type                  = var.instance_type
  instance_count                 = var.instance_count
  enable_encrypt_at_rest         = true
  enable_domain_endpoint_options = true
  advanced_security_enabled      = true

  tags = module.tags.tags
}

################################################################################
## opensearch serverless
################################################################################
module "opensearch_serverless" {
  source                       = "sourcefuse/arc-opensearch/aws"
  version                      = "1.0.3"
  enable_serverless            = true
  namespace                    = var.namespace
  environment                  = var.environment
  name                         = var.name
  enable_public_access         = true
  data_lifecycle_policy_rules  = local.data_lifecycle_policy_rules
  access_policy_rules          = local.access_policy_rules

  tags = module.tags.tags
}

Requirements

Name	Version
terraform	>= 1.5.0
aws	~> 5.0

Providers

Name	Version
aws	5.74.0

Modules

Name	Source	Version
opensearch	./modules/opensearch-domain	n/a
opensearch_serverless	./modules/opensearch-serverless	n/a

Resources

Name	Type
aws_caller_identity.current	data source

Inputs

Name	Description	Type	Default	Required
access_policies	Custom access policy for OpenSearch domain. If empty, default policy will be used	string	""	no
access_policy_rules	List of rules for the access policy.	list(object({ resource_type = string resource = list(string) permissions = list(string) }))	[]	no
advanced_security_enabled	Enable advanced security options (fine-grained access control)	bool	false	no
anonymous_auth_enabled	Enable anonymous authentication	bool	false	no
auto_software_update_enabled	Enable automatic software updates for OpenSearch	bool	false	no
auto_tune_cron_expression	Cron expression for Auto-Tune maintenance schedule	string	"0 1 * * ?"	no
auto_tune_desired_state	Desired state of Auto-Tune	string	"ENABLED"	no
auto_tune_duration_unit	Duration unit for Auto-Tune maintenance	string	"HOURS"	no
auto_tune_duration_value	Duration value for Auto-Tune maintenance	number	1	no
auto_tune_start_at	Start time for Auto-Tune maintenance	string	"2024-10-23T01:00:00Z"	no
availability_zone_count	The number of availability zones to use for zone awareness.	number	2	no
cognito_identity_pool_id	Cognito Identity Pool ID	string	""	no
cognito_user_pool_id	Cognito User Pool ID	string	""	no
create_access_policy	Flag to determine if access policy should be created.	bool	true	no
create_data_lifecycle_policy	Flag to determine if data lifecycle policy should be created.	bool	true	no
create_encryption_policy	Flag to determine if encryption policy should be created.	bool	true	no
custom_certificate_arn	ARN of the ACM certificate for the custom endpoint	string	""	no
custom_hostname	Custom domain name for the OpenSearch endpoint	string	""	no
data_lifecycle_policy_rules	Data lifecycle policy rules for the indices.	list(object({ indexes = list(string) retention = string }))	[ { "indexes": [ "*" ], "retention": "Unlimited" } ]	no
dedicated_master_count	Number of dedicated master instances	number	3	no
dedicated_master_enabled	Whether dedicated master is enabled	bool	false	no
dedicated_master_type	Instance type for the dedicated master node	string	"m5.large.search"	no
description	A description for the OpenSearch collection.	string	"OpenSearch collection domain for logs and search"	no
ebs_enabled	Whether EBS is enabled for the domain	bool	true	no
egress_rules	A list of egress rules for the security group.	list(object({ from_port = number to_port = number protocol = string cidr_blocks = list(string) }))	[]	no
enable_auto_tune	Enable Auto-Tune for the domain	bool	false	no
enable_cognito_options	Enable Cognito authentication for the OpenSearch domain	bool	false	no
enable_custom_endpoint	Enable custom domain endpoint	bool	false	no
enable_domain_endpoint_options	Enable custom domain endpoint options for the OpenSearch domain.	bool	false	no
enable_encrypt_at_rest	Enable encryption at rest for the OpenSearch domain.	bool	false	no
enable_off_peak_window_options	Enable off-peak window options for the domain	bool	false	no
enable_public_access	Enable public access for the OpenSearch collection. If false, private access will be used.	bool	false	no
enable_serverless	Enable OpenSearch Serverless. If true, creates the serverless module; if false, creates the standard module.	bool	false	no
enable_snapshot_options	Enable snapshot options for the domain	bool	false	no
enable_vpc_options	Enable VPC options for the OpenSearch domain.	bool	false	no
enable_zone_awareness	Enable zone awareness for the OpenSearch domain.	bool	false	no
encrypt_at_rest_enabled	Enable encryption at rest	bool	true	no
enforce_https	Force HTTPS on the OpenSearch endpoint	bool	true	no
engine_version	OpenSearch or Elasticsearch engine version	string	"OpenSearch_1.0"	no
environment	Name of the environment, i.e. dev, stage, prod	string	n/a	yes
ingress_rules	A list of ingress rules for the security group.	list(object({ from_port = number to_port = number protocol = string cidr_blocks = list(string) }))	[]	no
instance_count	Number of instances in the cluster	number	2	no
instance_type	Instance type for the OpenSearch domain	string	"m5.large.search"	no
internal_user_database_enabled	Enable internal user database for fine-grained access control	bool	true	no
iops	Provisioned IOPS for the volume	number	null	no
kms_key_id	KMS key ID for encryption at rest	string	""	no
log_publishing_enabled	Whether to enable the log publishing option.	bool	true	no
log_types	List of log types to publish to CloudWatch (Valid values: INDEX_SLOW_LOGS, SEARCH_SLOW_LOGS, ES_APPLICATION_LOGS, AUDIT_LOGS)	list(string)	[ "INDEX_SLOW_LOGS", "SEARCH_SLOW_LOGS" ]	no
master_user_arn	The ARN of the IAM role for fine-grained access control. Required if use_iam_arn_as_master_user is true.	string	""	no
master_user_name	Master user name for OpenSearch	string	"admin"	no
name	Name of the OpenSearch domain	string	n/a	yes
namespace	Namespace of the project, i.e. arc	string	n/a	yes
node_to_node_encryption_enabled	Enable node-to-node encryption	bool	true	no
off_peak_hours	Off-peak window start time (hours)	number	0	no
off_peak_minutes	Off-peak window start time (minutes)	number	0	no
retention_in_days	The number of days to retain log events in the log group	number	7	no
saml_options	Configuration block for SAML options in the OpenSearch domain.	object({ enabled = bool idp_entity_id = optional(string) idp_metadata_content = optional(string) roles_key = optional(string) session_timeout_minutes = optional(number) subject_key = optional(string) })	{ "enabled": false, "idp_entity_id": null, "idp_metadata_content": null, "roles_key": null, "session_timeout_minutes": null, "subject_key": null }	no
security_group_name	Name for the security group	string	""	no
snapshot_start_hour	Start hour for the automated snapshot	number	0	no
subnet_ids	List of subnet IDs for the OpenSearch domain	list(string)	[]	no
tags	Tags to apply to resources	map(string)	n/a	yes
throughput	Provisioned throughput for the volume	number	null	no
tls_security_policy	TLS security policy for HTTPS endpoints	string	"Policy-Min-TLS-1-2-PFS-2023-10"	no
type	The type of OpenSearch collection.	string	"TIMESERIES"	no
use_iam_arn_as_master_user	Set to true to use IAM ARN as the master user, false to create a master user.	bool	false	no
use_standby_replicas	Flag to enable or disable standby replicas.	bool	true	no
use_ultrawarm	Whether to enable UltraWarm nodes	bool	false	no
volume_size	EBS volume size in GB	number	20	no
volume_type	EBS volume type	string	"gp2"	no
vpc_id	ID of the VPC for OpenSearch domain	string	null	no
warm_count	Number of UltraWarm instances	number	2	no
warm_type	UltraWarm node instance type	string	"ultrawarm1.medium.search"	no
zone_awareness_enabled	Whether zone awareness is enabled	bool	true	no

Outputs

Name	Description
opensearch_domain_arn	The ARN of the OpenSearch domain.
opensearch_domain_endpoint	The endpoint of the OpenSearch domain.
opensearch_domain_id	The unique identifier for the OpenSearch domain.
opensearch_serverless_collection_arn	The ARN of the OpenSearch Serverless collection
opensearch_serverless_collection_id	The ID of the OpenSearch Serverless collection

Versioning

This project uses a .version file at the root of the repo which the pipeline reads from and does a git tag.

When you intend to commit to main, you will need to increment this version. Once the project is merged, the pipeline will kick off and tag the latest git commit.

Development

Prerequisites

• terraform
• terraform-docs
• pre-commit
• golang
• golint

Configurations

• Configure pre-commit hooks

  pre-commit install

Tests

• Tests are available in test directory
• Configure the dependencies

  cd test/
  go mod init github.com/sourcefuse/terraform-aws-refarch-opensearch
  go get github.com/gruntwork-io/terratest/modules/terraform

• Now execute the test

  go test -timeout  30m

Authors

This project is authored by:

• SourceFuse ARC Team