Terraform Enterprise (TFE) is a paid solution that extends the open source versions of terraform.
TFE makes it easier for teams to collaborate and govern Terraform changes and provides the support services that enterprises expect and is available as a SaaS or private install
- For teams of operators, it enables collaboration on infrastructure and a central service registry to publish and discover infrastructure modules.
- For organizations, it enables policy and governance to confidently provision at scale.
- SAML support for Single Sign on with TFE
- TFE can access your private resources (i.e. manage a vsphere environment in your DC)
- Customer managed, so it can meet compliance requirements such as PCI, SOC 1 & 2
- Logging to location of your choice
- Customizable Docker containers for TFE worker environments
Terraform Enterprise (TFE) is integrated with your version control system (VCS) provider. (i.e. GitHub, BitBucket, etc)
- When workspaces are linked to a VCS repository, TFE automatically initiates Terraform runs (plan & apply) when changes are committed to the specified branch.
- TFE makes code review easier by automatically predicting how pull requests will affect infrastructure.
- Publishing versions of a private Terraform module is as easy as pushing a tag to the module's git repository.
To use configurations stored as repositories in your VCS, TFE does several things:
-
Accesses a list of repositories, to let you search for repos when creating new workspaces.
-
Registers webhooks with your VCS provider, to get notified of new commits to a chosen branch.
-
Downloads the contents of a repository at a specific commit in order to run Terraform with that code.
-
When someone adds new commits to a branch in a repository linked to TFE, any workspaces based on that branch will begin a Terraform run.
- Usually a user must inspect the plan output and approve an apply
- You can also enable automatic applies on a per-workspace basis
- You can prevent automatic runs by locking a workspace
-
When someone submits a pull request/merge request to the "watched branch" from another branch in the same repository, TFE performs a speculative plan with the contents of the request and links to the results on the pull request's page.
- This helps you avoid merging PRs that cause plan failures.
Workspaces are how Terraform Enterprise (TFE) organizes your infrastructure. A workspace consists of:
- A Terraform configuration (usually retrieved from a VCS repo, but sometimes uploaded directly via API).
- Values for Terraform HCL and Environment variables used by the configuration
- Persistently stored state for the resources the configuration manages
- Historical record of states, runs and logs
- Role Based Access permissions for what teams can manage the repository
- The TFE UI only includes workspaces where your user account has at least read permissions.
- If the list is large, you can use the filter tools to control what you see
-
We recommend that organizations break down large monolithic Terraform configurations into smaller ones, then assign each configuration to its own workspace and delegate permissions and responsibilities for them.
-
We recommend naming related TFE workspaces and VCS repositories with the same name structure.
- A good strategy to start with is CLOUD-DEPT-COMPONENT-ENVIRONMENT[-REGION if appropriate]
- aws-marketing-app1-prod
- az-infra-network-dev-cus
- A good strategy to start with is CLOUD-DEPT-COMPONENT-ENVIRONMENT[-REGION if appropriate]
-
Managing smaller infrastructure components is the best way to take full advantage of TFE's governance and delegation features.
-
Example: your production environment's infrastructure might be divided into the following workspaces with separate teams to manage each
- networking configuration in workspace: aws-infra-networking-prod-us-east-2
- main application's configuration in workspace: aws-marketing-app1-prod
- monitoring configuration in workspace: aws-monitoring-app1-prod
This enables teams to make changes in parallel to re-use configurations to manage other environments (ex. marketing-app1-dev probably has a lot of the same resources as marketing-app1-prod)
Each workspace in TFE maintains its own queue of runs, and processes those runs in order.
- Whenever a new run is initiated, it's added to the end of the queue.
- If there's already a run in progress, the new run won't start until the current one has completely finished
- Runs that are waiting for other runs to finish are in a pending state
- A workspace might have any number of pending runs.
TFE enforces Terraform's division between plan and apply operations.
- It always plans first, saves the plan's output, and uses that output for the apply.
- In the default configuration, it waits for user approval before running an apply, but you can configure workspaces to automatically apply successful plans.
Terraform Enterprise (TFE) workspaces can set values for two kinds of variables:
- Terraform input variables - define the parameters of a Terraform configuration.
- TF variables set values to variables in your code in the same way that *.tfvars or -var does on the cli
- TF variable values are strings by default. To enter list or map values, click the variable's "HCL" checkbox
- Shell environment variables - required by many providers for credentials and other data.
- You can also set environment variables that affect Terraform's behavior, like TF_LOG.
- To protect sensitive variables, click the "Sensitive" checkbox next to the variable (visible when editing). Marking a variable as sensitive prevents anybody (including you) from viewing its value in TFE's UI or API.
- TFE encrypts ALL variable values securely using Hashicorp Vault's transit backend prior to saving them. This ensures that no out-of-band party can read these values without proper authorization.
Terraform Enterprise required credentials into your cloud provider, so that it can provision infrastructure on your behalf
-
Both AWS and Azure store cloud access credentials in environment variables and require that the be set in TFE. See the cloud creds document for detailed instructions on creating the correct cloud credentials.
-
AWS provider requires the following 2 variables be set in each aws based TFE workspace environment variables section
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY (sensitive)
For AWS accounts, it is recommended to create an IAM administrative user with administrative privileges that
is dedicated to terraform and provide the access keys to each workspace. cloud creds
-
Azure provider requires the following 4 variables be set in each azure based TFE workspace environment variables section
- ARM_SUBSCRIPTION_ID
- ARM_CLIENT_ID
- ARM_TENANET_ID
- ARM_CLIENT_SECRET (sensitive)
In Azure accounts, it is recommended to create a principal account id for terraform using the following
cloud creds docs
-
VMWare vsphere provider requires the following Environment variables be set
- VSPHERE_USER
- VSPHERE_PASSWORD
- VSPHERE_SERVER
The TFE settings tab for a workspace allows certain parameters to be adjusted after workspace creation. You can change things like
- workspace name
- whether to automatically apply runs
- the specific version of terraform that you want to use
- the working directory of your repo for the terraform project
- workspace locks
- version control settings and VCS branch to use
- ability to destroy or delete the workspace
Terraform Enterprise (TFE) workspaces can only be accessed by users with the correct permissions.
- You can manage permissions for a workspace on a per-team basis.
- When a workspace is created, the only team able to access it is the owners team, with full admin permissions.
- The special administrative team called owners can never be removed from a workspace.
- Teams can have any of the following organizational level permissions
- Manage Policies (Allow members to create, edit, and delete the organization's Sentinel policies and override soft-mandatory policy checks)
- Manage Workspaces (Allow members to create and administrate all workspaces within the organization)
- Manage VCS Settings (Allow members to manage the organization's VCS Providers and SSH keys)
- Teams can have one of the following workspace level
permissions
- read (ability to see the workspace, its runs, states and configuration)
- plan (Can do everything the read access level can do plus: Create runs)
- write (all read privs + approve runs, lock workspaces & edit variables)
- admin (all write privs + delete workspace, change vcs settings & change permissions)
- A common permission model is to give engineers read access and give operations staff (or fire-ids) write access
Terraform Enterprise (TFE)'s private module registry helps you share Terraform modules across your organization.
- Support for module versioning
- a searchable and filterable list of available modules
- a configuration designer to help you build new workspaces faster.
- TFE can automatically access your private modules during Terraform runs
module "vpc" {
source = "app.terraform.io/example_corp/vpc/aws"
version = "1.0.4"
}
-
Private modules must be named with the following syntax terraform-PROVIDER-NAME
- PROVIDER is the main provider where it creates that infrastructure
- NAME reflects the type of infrastructure the module manages
- The segment can contain additional hyphens. ex: terraform-google-vault or terraform-aws-ec2-instance.
-
Use semantic version tags (x.y.z) for releases.
- The registry uses tags to identify module versions.
- Release tag names can optionally be prefixed with a v (v1.2.3)
-
To release a new version of a module, push a new version tag to its VCS repository. The registry will automatically import the new version.
-
To access TFE modules from terraform CLI, configure a credentials block in your CLI configuration file (.terraformrc) with a valid TFE API TOKEN
# $HOME/.terraformrc
credentials "app.terraform.io" {
token = "xxxxxx.atlasv1.zzzzzzzzzzzzz"
}