Spot Ocean k8s Virtual Node Group Terraform Module

Spotinst Terraform Module to integrate existing k8s node groups with Ocean launchspec/Virtual Node group (VNG)

Usage

Note: This module will automatically import any tags defined in the AWS provider default_tags

provider "spotinst" {
  token   = "redacted"
  account = "redacted"
}

module "ocean-aws-k8s" {
  ...
}

## Create Ocean Virtual Node Group (launchspec) ##
module "ocean-aws-k8s-vng_stateless" {
  source = "spotinst/ocean-aws-k8s-vng/spotinst"

  name = "stateless" # Name of VNG in Ocean
  ocean_id = module.ocean-aws-k8s.ocean_id
  
  #ami_id = "" # Can change the AMI
  labels = [{key="type",value="stateless"}]
}

## Create additional Ocean Virtual Node Group (launchspec) ##
module "ocean-aws-k8s-vng_gpu" {
  source = "spotinst/ocean-aws-k8s-vng/spotinst"

  name = "gpu"  # Name of VNG in Ocean
  ocean_id = module.ocean-aws-k8s.ocean_id
  
  labels = [{key="type",value="gpu"}]
  taints = [{key="type",value="gpu",effect="NoSchedule"}]
  
  #instance_types = ["g4dn.xlarge","g4dn.2xlarge"] # Limit VNG to specific instance types
  spot_percentage = 50 # Change the spot %
  draining_timeout = 300 
}

## Create Ocean Virtual Node Group (launchSpec) with instance_type_filters ##
module "ocean-aws-k8s-vng" {
  source  = "spotinst/ocean-aws-k8s-vng/spotinst"
  ocean_id = module.ocean-aws-k8s.ocean_id
  name = "test-vng"
  instance_types_filters_enable                   =   true
  instance_types_filters_categories               =   ["Accelerated_computing", "Compute_optimized"]
  instance_types_filters_disk_types               =   ["NVMe", "EBS"]
  instance_types_filters_exclude_families         =   ["t2","R4*"]
  instance_types_filters_exclude_metal            =   true
  instance_types_filters_hypervisor               =   ["nitro"]
  instance_types_filters_include_families         =   ["c5*", "g5"]
  instance_types_filters_is_ena_supported         =   true
  instance_types_filters_max_gpu                  =   4
  instance_types_filters_min_gpu                  =   0
  instance_types_filters_max_memory_gib           =   16
  instance_types_filters_max_network_performance  =   20
  instance_types_filters_max_vcpu                 =   16
  instance_types_filters_min_enis                 =   2
  instance_types_filters_min_memory_gib           =   8
  instance_types_filters_min_network_performance  =   2
  instance_types_filters_min_vcpu                 =   2
  instance_types_filters_root_device_types        =   ["ebs"]
  instance_types_filters_virtualization_types     =   ["hvm"]
  images                                          =   [{image_id="ami-123456"},{image_id="ami-67890"}]
  block_device_mappings                           =   [{
    device_name                                   =   "/dev/xvda"
    encrypted                                     =   false
    volume_type                                   =   "gp3"
  }]
  dynamic_volume_size                             =   {
    base_size                                     =   50
    resource                                      =   "CPU"
    size_per_resource_unit                        =   20
  }
  ephemeral_storage_device_name                   =   "/dev/xvdb"
  preferred_spot_types                            =   ["m4.xlarge", "c3.large"]
  preferred_od_types                              =   ["t3.medium", "c4.large"]

}

~> You must configure the spotinst_ocean_aws resource. Ensure spotinst_ocean_aws resource (defined in ocean-aws-k8s module) is defined before this module as the ocean_id is needed.

Providers

Name	Version
spotinst/spotinst	>= 1.95

Modules

• ocean-aws-k8s - Creates Ocean Cluster Doc
• ocean-controller - Create and installs spot ocean controller pod Doc
• ocean-aws-k8s-vng - (Optional) Add custom virtual node groups Doc

Documentation

If you're new to Spot and want to get started, please checkout our Getting Started guide, available on the Spot Documentation website.

Getting Help

We use GitHub issues for tracking bugs and feature requests. Please use these community resources for getting help:

• Ask a question on Stack Overflow and tag it with terraform-spotinst.
• Join our Spot community on Slack.
• Open an issue.

Community

• Slack
• Twitter

Contributing

Please see the contribution guidelines.

Resources

Name	Type
ocean-aws-k8s-vng	resource

Inputs

Name	Description	Type	Default	Required
ocean_id	The Ocean cluster identifier. Required for Launch Spec creation.	string	null	yes
name	Set launch specification name.	string	null	no
instance_types_filters_enable	'instance_types_filters_enable' to be set to true to have instance_types_filters configured in the virtual node group.	boolean	null	no
instance_types_filters	The instance types that match with all filters compose the Virtual Node Group's instanceTypes parameter. The architectures that come from the Virtual Node Group's images will be taken into account when using this parameter. Cannot be configured together with Virtual Node Group's instanceTypes and with the Cluster's whitelist/blacklist/filters.	object({ categories = list(string) disk_types = list(string) exclude_families = list(string) exclude_metal = bool hypervisor = list(string) include_families = list(string) is_ena_supported = bool max_gpu = number min_gpu = number max_memory_gib = number max_network_performance = number max_vcpu = number min_enis = number min_memory_gib = number min_network_performance = number min_vcpu = number root_device_types = list(string) virtualization_types = list(string) })	null	no
images	You can configure VNG with either the imageId or images objects, but not both simultaneously. For each architecture type (amd64, arm64) only one AMI is allowed. Valid values: null, or an array with at least one element.	list(object({ image_id = string }))	null	no
block_device_mappings	block_device_mapping object	list(object({ device_name = string delete_on_termination = bool encrypted = bool kms_key_id = string snapshot_id = string volume_type = string iops = number volume_size = number throughput = number }))	[]	no
dynamic_volume_size	dynamic_volume_size object	object({ base_size = number size_per_resource_unit = number resource = string })	null	no
ephemeral_storage_device_name	ephemeral_storage_device_name	string	null	no
draining_timeout	The configurable amount of time that Ocean will wait for the draining process to complete before terminating an instance. If you have not defined a draining timeout, the default of 300 seconds will be used.	number	300	no
spot_percentage	The desired percentage of the Spot instances out of all running instances for this VNG. Only available when the field is not set in the cluster directly (cluster.strategy.spotPercentage).	number	100	no
utilize_commitments	When set as ‘true’, if savings plans commitments have available capacity, Ocean will utilize them alongside RIs (if exist) to maximize cost efficiency. If the value is set as 'null', it will automatically be inherited from the cluster level.	bool	false	no
utilize_reserved_instances	When set as ‘true’, if reserved instances exist, Ocean will utilize them before launching spot instances. If the value is set as 'null', it will automatically be inherited from the cluster level.	bool	true	no
respect_pdb	Default: false. During the roll, if the parameter is set to True we honor PDB during the instance replacement.	bool	null	no
reserved_enis	Default: 0. Specifies the count of ENIs to reserve per instance type for scaling purposes.	number	0	no

Outputs

Name	Description
virtual_node_group_id	The virtual node group ID