Azure Kubernetes Service (AKS)

This is an opinionated terraform module to bootstrap an AKS Cluster using Terraform.

Features enabled:

• Logging using Azure Monitor and Azure Log Analytics
• Cluster Autoscaler
• Metrics server configured and fully functional
• Configurable default node pool autoscaling
• Configurabel secondary node pool regular or spot instances
• Container registry integration

Usage

You can find a fully functional, production-ready example on the examples/ folder.

Requirements

Name	Version
terraform	>= 1.2
azurerm	>= 3.27
tls	>= 3.1

Providers

Name	Version
azurerm	>= 3.27
tls	>= 3.1

Configurations

• Configure Terraform for Azure

We assumed that you have setup service principal's credentials in your environment variables like below:

export ARM_SUBSCRIPTION_ID="<azure_subscription_id>"
export ARM_TENANT_ID="<azure_subscription_tenant_id>"
export ARM_CLIENT_ID="<service_principal_appid>"
export ARM_CLIENT_SECRET="<service_principal_password>"

On Windows Powershell:

$env:ARM_SUBSCRIPTION_ID="<azure_subscription_id>"
$env:ARM_TENANT_ID="<azure_subscription_tenant_id>"
$env:ARM_CLIENT_ID="<service_principal_appid>"
$env:ARM_CLIENT_SECRET="<service_principal_password>"

Import the Resource Group into Terraform

1. Add the following resource block and save the file

resource "azurerm_resource_group" "main" {}

2. Run the az group list command to get the subscription id.
3. Import your resource group into Terraform.

terraform import azurerm_resource_group.main /subscriptions/<azure_subscription_id>/resourceGroups/<RESOURCE_GROUP>

4. Add to the main.tf file so it looks like the code below (fill in with your resource group and location):

resource "azurerm_resource_group" "main" {
  name = "<RESOURCE_GROUP>"
  location = "<LOCATION>"
  tags = {
    environment = "demo"
  }
}

Inputs

Name	Description	Type	Default	Required
prefix	The prefix for the resources created in the specified Azure Resource Group.	string	n/a	yes
resource_group_name	The resource group name to be imported.	string	n/a	yes
aci_connector_linux_enabled	Enable Virtual Node pool.	bool	false	no
aci_connector_linux_subnet_name	aci_connector_linux subnet name.	string	null	no
admin_username	The username of the local administrator to be created on the Kubernetes cluster. Set this variable to null to turn off the cluster's linux_profile. Changing this forces a new resource to be created.	string	null	no
agents_tags	A mapping of tags to assign to the Node Pool.	map(string)	{}	no
agents_type	The type of Node Pool which should be created. Possible values are AvailabilitySet and VirtualMachineScaleSets.	string	VirtualMachineScaleSets	no
api_server_authorized_ip_ranges	The IP ranges to allow for incoming traffic to the server nodes.	string	null	no
azure_policy_enabled	Enable Azure Policy Addon.	bool	false	no
client_id	The Client ID (appId) for the Service Principal used for the AKS deployment.	string	""	no
client_secret	The Client Secret (password) for the Service Principal used for the AKS deployment.	string	""	no
cluster_log_analytics_workspace_name	The name of the Analytics workspace.	string	null	no
cluster_name	The name for the AKS resources created in the specified Azure Resource Group. This variable overwrites the 'prefix' var (The 'prefix' var will still be applied to the dns_prefix if it is set).	string	null	no
disk_encryption_set_id	The ID of the Disk Encryption Set which should be used for the Nodes and Volumes. Changing this forces a new resource to be created.	string	null	no
http_application_routing_enabled	Enable HTTP Application Routing Addon (forces recreation).	bool	false	no
identity_ids	Specifies a list of User Assigned Managed Identity IDs to be assigned to this Kubernetes Cluster.	list(string)	null	no
identity_type	The type of identity used for the managed cluster. Conflict with client_id and client_secret. Possible values are SystemAssigned, UserAssigned, SystemAssigned, UserAssigned(to enable both). If UserAssigned or SystemAssigned, UserAssigned is set, an identity_ids must be set as well.	string	SystemAssigned	no
ingress_application_gateway_enabled	Whether to deploy the Application Gateway ingress controller to this Kubernetes Cluster.	bool	false	no
ingress_application_gateway_name	The name of the Application Gateway to be used or created in the Nodepool Resource Group, which in turn will be integrated with the ingress controller of this Kubernetes Cluster.	string	null	no
ingress_application_gateway_subnet_cidr	The subnet CIDR to be used to create an Application Gateway, which in turn will be integrated with the ingress controller of this Kubernetes Cluster.	string	null	no
ingress_application_gateway_subnet_id	The ID of the subnet on which to create an Application Gateway, which in turn will be integrated with the ingress controller of this Kubernetes Cluster.	string	null	no
key_vault_secrets_provider_enabled	Whether to use the Azure Key Vault Provider for Secrets Store CSI Driver in an AKS cluster. For more details: https://docs.microsoft.com/en-us/azure/aks/csi-secrets-store-driver".	bool	false	no
kubernetes_version	Specify which Kubernetes release to use. The default used is the latest Kubernetes version available in the region.	string	null	no
local_account_disabled	If true local accounts will be disabled. Defaults to false. See the documentation for more information.	bool	false	no
location	Location of cluster, if not defined it will be read from the resource-group.	string	null	no
log_analytics_solution_id	Existing azurerm_log_analytics_solution ID. Providing ID disables creation of azurerm_log_analytics_solution.	string	null	no
log_analytics_workspace	Existing azurerm_log_analytics_workspace to attach azurerm_log_analytics_solution. Providing the config disables creation of azurerm_log_analytics_workspace.	object(string)	null	no
log_analytics_workspace_enabled	Enable the integration of azurerm_log_analytics_workspace and azurerm_log_analytics_solution: https://docs.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-onboard.	bool	true	no
log_analytics_workspace_resource_group_name	Resource group name to create azurerm_log_analytics_solution.	string	null	no
log_analytics_workspace_sku	The SKU (pricing level) of the Log Analytics workspace. For new subscriptions the SKU should be set to PerGB2018.	string	PerGB2018	no
log_retention_in_days	The retention period for the logs in days.	number	30	no
maintenance_window	Maintenance configuration of the managed cluster.	object(string)	null	no
microsoft_defender_enabled	Is Microsoft Defender on the cluster enabled? Requires var.log_analytics_workspace_enabled to be true to set this variable to true.	bool	false	no
net_profile_dns_service_ip	IP address within the Kubernetes service address range that will be used by cluster service discovery (kube-dns). Changing this forces a new resource to be created.	string	null	no
net_profile_docker_bridge_cidr	IP address (in CIDR notation) used as the Docker bridge IP address on nodes. Changing this forces a new resource to be created.	string	null	no
net_profile_outbound_type	The outbound (egress) routing method which should be used for this Kubernetes Cluster. Possible values are loadBalancer and userDefinedRouting.	string	loadBalancer	no
net_profile_pod_cidr	The CIDR to use for pod IP addresses. This field can only be set when network_plugin is set to kubenet. Changing this forces a new resource to be created.	string	null	no
net_profile_service_cidr	The Network Range used by the Kubernetes service. Changing this forces a new resource to be created.	string	null	no
network_plugin	Network plugin to use for networking.	string	kubenet	no
network_policy	Sets up network policy to be used with Azure CNI. Network policy allows us to control the traffic flow between pods. Currently supported values are calico and azure. Changing this forces a new resource to be created.	string	null	no
node_resource_group	The auto-generated Resource Group which contains the resources for this Managed Kubernetes Cluster. Changing this forces a new resource to be created.	string	null	no
oidc_issuer_enabled	Enable or Disable the OIDC issuer URL. Defaults to false.	bool	false	no
only_critical_addons_enabled	Enabling this option will taint default node pool with CriticalAddonsOnly=true:NoSchedule taint. Changing this forces a new resource to be created.	bool	false	no
open_service_mesh_enabled	Is Open Service Mesh enabled? For more details, please visit Open Service Mesh for AKS.	bool	false	no
orchestrator_version	Specify which Kubernetes release to use for the orchestration layer. The default used is the latest Kubernetes version available in the region	string	null	no
pod_subnet_id	The ID of the Subnet where the pods in the default Node Pool should exist. Changing this forces a new resource to be created.	string	null	no
private_cluster_enabled	If true cluster API server will be exposed only on internal IP address and available only in cluster vnet.	bool	false	no
private_cluster_public_fqdn_enabled	Specifies whether a Public FQDN for this Private Cluster should be ad	bool	false	no
private_dns_zone_id	Either the ID of Private DNS Zone which should be delegated to this Cluster, System to have AKS manage this or None. In case of None you will need to bring your own DNS server and set up resolving, otherwise cluster will have issues after provisioning.	string	null	no
public_ssh_key	A custom ssh key to control access to the AKS cluster. Changing this forces a new resource to be created.	string	""	no
rbac_aad_admin_group_object_ids	Object ID of groups with admin access.	list(string)	null	no
rbac_aad_azure_rbac_enabled	Is Role Based Access Control based on Azure AD enabled.	string	null	no
rbac_aad_client_app_id	The Client ID of an Azure Active Directory Application.	bool	false	no
rbac_aad_managed	Is the Azure Active Directory integration Managed, meaning that Azure will create/manage the Service Principal used for integration.	bool	false	no
rbac_aad_server_app_id	The Server ID of an Azure Active Directory Application.	string	null	no
rbac_aad_server_app_secret	The Server Secret of an Azure Active Directory Application.	string	null	no
rbac_aad_tenant_id	The Tenant ID used for Azure Active Directory Application. If this isn't specified the Tenant ID of the current Subscription is used.	string	null	no
role_based_access_control_enabled	Enable Role Based Access Control.	bool	false	no
secret_rotation_enabled	Is secret rotation enabled? This variable is only used when key_vault_secrets_provider_enabled is true and defaults to false.	bool	false	no
secret_rotation_interval	The interval to poll for secret rotation. This attribute is only set when secret_rotation is true.	string	2m	no
sku_tier	The SKU Tier that should be used for this Kubernetes Cluster. Possible values are Free and Paid.	string	Free	no
tags	Any tags that should be present on the AKS cluster resources.	map(string)	{}	no
vnet_subnet_id	The ID of a Subnet where the Kubernetes Node Pool should exist. Changing this forces a new resource to be created.	string	null	no
workload_identity_enabled	Enable or Disable Workload Identity.	bool	false	no
default_node_pool_agents_availability_zones	A list of Availability Zones across which the Node Pool should be spread. Changing this forces a new resource to be created.	string	null	no
default_node_pool_node_count	The number of Agents that should exist in the default node Pool. Please set agents_count null while enable_auto_scaling is true to avoid possible agents_count changes.	number	1	no
default_node_pool_labels	A map of Kubernetes labels which should be applied to nodes in the Default Node Pool. Changing this forces a new resource to be created.	map(string)	{}	no
default_node_pool_max_count	Maximum number of nodes in a default node pool.	number	null	no
default_node_pool_max_pods	The maximum number of pods that can run on default ndoe pool. Changing this forces a new resource to be created.	number	null	no
default_node_pool_min_count	Minimum number of nodes in a default node pool.	number	null	no
default_node_pool_name	The default Azure AKS default ndoe pool name.	string	default	no
default_node_pool_size	The default virtual machine size for the Kubernetes default node pool.	string	Standard_D2s_v3	no
default_node_pool_enable_auto_scaling	Enable default node pool autoscaling.	bool	true	no
default_node_pool_enable_host_encryption	Enable Host Encryption for default node pool.	bool	false	no
default_node_pool_enable_node_public_ip	Should nodes in default Node Pool have a Public IP Address? Defaults to false.	bool	false	no
default_node_pool_os_disk_size_gb	Disk size of default node pool in GBs.	number	30	no
default_node_pool_os_disk_type	The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Changing this forces a new resource to be created.	string	Managed	no
default_node_pool_ultra_ssd_enabled	Used to specify whether the UltraSSD is enabled in the Default Node Pool.	bool	false	no
secondary_node_pool_node_count	The number of Agents that should exist in the Secondary node Pool. Please set agents_count null while enable_auto_scaling is true to avoid possible agents_count changes.	number	1	no
secondary_node_pool_node_count	The number of Agents that should exist in the Secondary node Pool. Please set agents_count null while enable_auto_scaling is true to avoid possible agents_count changes.	number	1	no
secondary_node_pool_max_count	Maximum number of nodes in a secondary node pool.	number	null	no
secondary_node_pool_max_pods	The maximum number of pods that can run on secondary node pool. Changing this forces a new resource to be created.	number	null	no
secondary_node_pool_min_count	Minimum number of nodes in a secondary node pool.	number	null	no
secondary_node_pool_name	The secondary Azure AKS default ndoe pool name.	string	n/a	yes
secondary_node_pool_size	The default virtual machine size for the Kubernetes secondary node pool.	string	Standard_D2s_v3	no
secondary_node_pool_enable_auto_scaling	Enable secondary node pool autoscaling.	bool	true	no
secondary_node_pool_enable_host_encryption	Enable Host Encryption for secondary node pool.	bool	false	no
secondary_node_pool_enable_host_encryption	Enable Host Encryption for secondary node pool.	bool	false	no
secondary_node_pool_enable_node_public_ip	Should nodes in secondary Node Pool have a Public IP Address? Defaults to false.	bool	false	no
secondary_node_pool_os_disk_size_gb	Disk size of secondary node pool in GBs.	number	30	no
secondary_node_pool_os_disk_type	The type of disk which should be used for the Operating System. Possible values are Ephemeral and Managed. Changing this forces a new resource to be created.	string	Managed	no
secondary_node_pool_ultra_ssd_enabled	Used to specify whether the UltraSSD is enabled in the Secondary Node Pool.	bool	false	no
secondary_node_pool_priority	The priority for Spot nodes in the Node Pool. Possible values are Spot and Regular.	string	Spot	no
secondary_node_pool_eviction_policy	The eviction policy for Spot nodes in the Node Pool. Possible values are Delete and Deallocate.	string	Delete	no
secondary_node_pool_spot_max_price	The maximum price per hour that you are willing to pay for Spot nodes. Changing this forces a new resource to be created.	number	0.5	no
acr_name	The name of the container registry.	string	""	no
acr_sku	The SKU of the container registry.	Basic	Basic	no
acr_admin_enabled	The admin user enabled status of the container registry.	bool	true	no
acr_zone_redundancy_enabled	Zone redundancy enabled.	bool	false	no

Outputs

Name	Description
container_registry_name	value of the container registry name.
container_registry_login_server	value of the container registry login server.
azurerm_kubernetes_cluster_main_object_id	value of the object id of the service principal.
azurerm_kubernetes_cluster_main_kube_config_host	value of the host of the kube config.
azurerm_kubernetes_cluster_main_kube_config_client_certificate	value of the client certificate of the kube config.
azurerm_kubernetes_cluster_main_kube_config_client_key	value of the client key of the kube config.
azurerm_kubernetes_cluster_main_kube_config_cluster_ca_certificate	value of the cluster ca certificate of the kube config.
azurerm_kubernetes_cluster_main_kube_config_cluster_raw	value of the cluster raw of the kube config.
aks_id	The azurerm_kubernetes_cluster's id.
aks_name	The aurerm_kubernetes-cluster's name.
azure_policy_enabled	The azurerm_kubernetes_cluster's azure_policy_enabled argument.
azurerm_log_analytics_workspace_id	The id of the created Log Analytics workspace.
azurerm_log_analytics_workspace_name	The name of the created Log Analytics workspace.
azurerm_log_analytics_workspace_primary_shared_key	Specifies the workspace key of the log analytics workspace.