Skip to content

Commit

Permalink
Merge pull request #281 from bitovi/doc/fixes
Browse files Browse the repository at this point in the history
Various documentation fixes
  • Loading branch information
arm4b authored Aug 22, 2022
2 parents 99cd1d1 + d30012a commit 947b4d2
Show file tree
Hide file tree
Showing 17 changed files with 124 additions and 154 deletions.
26 changes: 13 additions & 13 deletions docs/cloud-configuration/configuration-aws.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Note from the developers: We are currently in the process of moving our documentation and so the below documentation is only partially correct. For more information on this tool please checkout our [plugin documentation](https://github.com/bitops-plugins/aws)
> ⚠️ Note from the developers: We are currently in the process of moving our documentation and so the below documentation is only partially correct. For more information on this tool please checkout our [plugin documentation](https://github.com/bitops-plugins/aws).
# AWS

`bitops.config.yaml` is not yet supported for AWS ([TODO](https://github.com/bitovi/bitops/issues/15)). All configuration must be done with environment variables
> ⚠️ `bitops.config.yaml` is not yet supported for AWS ([TODO](https://github.com/bitovi/bitops/issues/15)). All configuration must be done with environment variables.
## Configuration

Expand All @@ -13,7 +13,7 @@ Note from the developers: We are currently in the process of moving our document
* **default:** `""`
* **required:** yes

AWS Access. See [AWS official documentation](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)
Specifies an AWS access key associated with an IAM user or role. See [AWS official documentation](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)

-------------------
### aws_secret_access_key
Expand All @@ -22,24 +22,24 @@ AWS Access. See [AWS official documentation](https://docs.aws.amazon.com/general
* **default:** `""`
* **required:** yes

AWS Access. See [AWS official documentation](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)
Specifies the secret key associated with the access key. This is essentially the "password" for the access key. See [AWS official documentation](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)

-------------------
### aws_session_token
### aws_default_region
* **BitOps Property:** [TODO](https://github.com/bitovi/bitops/issues/15)
* **Environment Variable:** `AWS_SESSION_TOKEN`
* **Environment Variable:** `AWS_DEFAULT_REGION`
* **default:** `""`
* **required:** no
* **required:** yes

AWS Access. See [AWS official documentation](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)
Specifies the AWS Region to send the request to. See [AWS official documentation](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)

-------------------
### aws_default_region
### aws_session_token
* **BitOps Property:** [TODO](https://github.com/bitovi/bitops/issues/15)
* **Environment Variable:** `AWS_DEFAULT_REGION`
* **Environment Variable:** `AWS_SESSION_TOKEN`
* **default:** `""`
* **required:** yes
* **required:** no

AWS Access. See [AWS official documentation](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)
Specifies the session token value that is required if you are using temporary security credentials that you retrieved directly from AWS STS operations. See [AWS official documentation](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys)

-------------------
-------------------
12 changes: 6 additions & 6 deletions docs/configuration-base.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Base Configuration

Each deployment tool is traditionally controlled with a set of cli arguments. Instead of defining arguments within your pipeline configuration, they can instead either be defined with environment variables or in a `bitops.config.yml` file. While the core schema for a `bitops.config.yml` file is common between tools, the specific properties and environment variable equivilants vary from tool to tool.
Each deployment tool is traditionally controlled with a set of CLI arguments. Instead of defining arguments within your pipeline configuration, they can instead either be defined with environment variables or in a `bitops.config.yaml` file. While the core schema for a `bitops.config.yaml` file is common between tools, the specific properties and environment variable equivalents vary from tool to tool.

> For more information on tool configuration, see [plugins](plugins.md).
Expand All @@ -12,20 +12,20 @@ $tool
cli: {}
options: {}
```

* `$tool` - identifies the deployment tool
* `cli` - object that contains cli arguments
* `options` - object that offers additional control over how a tool executes
* `cli` - object that contains CLI arguments
* `options` - an object that offers additional control over how a tool executes

## Arbitrary Environment Variables
During the docker run command, you can specify a ENV var and it will be accessible during all processing stages of BitOps.
During the docker run command, you can specify an ENV var and it will be accessible during all processing stages of BitOps.

## Common Configuration
There are some global configuration options that are shared among all tools and cloud providers during a BitOps run. These are set via environment variables

-------------------
### environment
* **Environment Variable:** `BITOPS_ENVIRONMENT`
* **default:** `""`
* **required:** yes

Each BitOps run is done against a single environment. This property tells BitOps which environment to run. For more information on environments, see [Operations Repo Structure](operations-repo-structure.md#environment-directories).
Expand All @@ -36,7 +36,7 @@ Each BitOps run is done against a single environment. This property tells BitOps
* **default:** `""`
* **required:** no

Base64 encoded `kubeconfig` file. Allows deployment tools to interact with a kubernetes cluster
Base64 encoded `kubeconfig` file. Allows deployment tools to interact with a Kubernetes cluster.

-------------------
## Cloud Providers
Expand Down
15 changes: 7 additions & 8 deletions docs/default-environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Suppose we are working with an operations repo that is exlusviely terraform. We
└── bitops.config.yaml
└── test.auto.tfvars
```
When `$BITOPS_ENVIRONMENT` is set to `production`, `_default/` will be merged in to `production/` at runtime to produce a directory structure that looks like
When `$BITOPS_ENVIRONMENT` is set to `production`, `_default/` will be merged into `production/` at runtime to produce a directory structure that looks this way:
```
├── _default
│   └── terraform
Expand All @@ -33,7 +33,7 @@ When `$BITOPS_ENVIRONMENT` is set to `production`, `_default/` will be merged in
└── test.auto.tfvars
```

Things get more complex when files exist in both the `_default` and `active` environment share the same name. This is why we have file mergers.
Things get more complex when files that exist in both the `_default` and `active` environment share the same name. This is why we have file mergers.


## File Mergers [TODO](https://github.com/bitovi/bitops/issues/3)
Expand All @@ -45,7 +45,7 @@ Files that only exist in the `_default` environment will be copied over.
`.tf` files from the `_default` environment that share its name and path with a file in the active environment will both be in the resulting directory with the active environment name having a suffix added to it.

#### Example
Before default merge
Before default merge:
```
├── _default
│   └── terraform
Expand All @@ -55,7 +55,7 @@ Before default merge
└── bitops.config.yaml
└── main.tf
```
After default merge
After default merge:
```
├── _default
│   └── terraform
Expand Down Expand Up @@ -109,7 +109,7 @@ Files that only exist in the `_default` environment will be copied over.
Files from the `_default` environment that share its name and path will be merged.

### values.yaml (Helm)
helm has built in support for merging multiple `values.yaml` files. BitOps will look for files in the following locations and pass them in to helm with with the `-f` in the same order they are found
Helm has built in support for merging multiple `values.yaml` files. BitOps will look for files in the following locations and pass them in to helm with the `-f` in the same order they are found:

1. Active environment's `values.yaml`
2. Default environment's `values.yaml`
Expand Down Expand Up @@ -139,10 +139,9 @@ The following operations repo structure
```
Will produce the following `helm install` command
```
helm install my-first-chart/
helm install \
$HELM_RELEASE_NAME
my-first-chart/
$HELM_RELEASE_NAME \
my-first-chart \
-f test/helm/my-first-chart/values.yaml \
-f default/helm/my-first-chart/values.yaml \
-f test/helm/my-first-chart/values-versions.yaml \
Expand Down
17 changes: 8 additions & 9 deletions docs/development-local.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
At this point you should have a repo for your custom BitOps image and have it open in your favorite editor. For more information on setting up your [custom BitOps image](custom-image.md)
At this point, you should have a repo for your custom BitOps image and have it open in your favorite editor. For more information on setting up your [custom BitOps image](custom-image.md)


## Create a plugins.dockerfile
Create a file in the root level of BitOps named `Dockerfile.plugins` with the content;

`FROM bitovi/bitops:2.0.0`
## Create plugins Dockerfile
Create a file in the root level of BitOps named `Dockerfile.plugins` with the following content:

```
FROM bitovi/bitops:base
```

## Build BitOps (with plugins)
`docker build -t bitops:latest . -f Dockerfile.plugins`

`docker build -t bitops:local . -f Dockerfile.plugins`

## Run BitOps
```
Expand All @@ -19,10 +19,9 @@ docker run \
-e AWS_ACCESS_KEY_ID=your-aws-access-key \
-e AWS_SECRET_ACCESS_KEY=your-aws-secret-access-key \
-v /path/to/operations_repo:/opt/bitops_deployment \
bitops:latest
bitops:local
```


# Benefits to local testing
- configuration validation
- rapid testing of new features
Expand Down
2 changes: 0 additions & 2 deletions docs/development/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,6 @@ Packages natively available in a running BitOps container:
> [Dockerfile](https://github.com/bitovi/bitops/blob/main/Dockerfile)
> and [requirements.txt](https://github.com/bitovi/bitops/blob/main/requirements.txt).
### BitOps Code Structure

The BitOps repo is fairly straightforward in its current implementation.
See `scripts/plugins/` written in Python for most of the logic responsible for running the pipelines that tie together deployment tools like `aws`, `ansible`, `terraform` which are called BitOps plugins.

Expand Down
13 changes: 7 additions & 6 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ docker run \
bitovi/bitops:latest
```

### Fetch kubeconfig from eks
If you has a cluster arn of `arn:aws:eks:us-east-1:111122223333:cluster/my-cluster`, you would use the following configuration
### Fetch kubeconfig from EKS
If you have a cluster arn of `arn:aws:eks:us-east-1:111122223333:cluster/my-cluster`, you would use the following configuration:
```
docker run \
-e BITOPS_ENVIRONMENT="dev" \
Expand All @@ -51,7 +51,7 @@ bitovi/bitops:latest
```

### Force skip over ansible
If there is a `dev/ansible/` directory, ansible execution can be skipped with `SKIP_DEPLOY_ANSIBLE`
If there is a `./dev/ansible/` directory, ansible execution can be skipped with `SKIP_DEPLOY_ANSIBLE=true`:
```
docker run \
-e BITOPS_ENVIRONMENT="dev" \
Expand All @@ -75,13 +75,14 @@ docker run \
bitovi/bitops:latest
```

## Use your local BitOps
## Use your local BitOps code
If you'd like to override the BitOps code with your own version, see how to do that with the `$BITOPS_HOME` example:
```
docker run
-e AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION \
-e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-e BITOPS_ENVIRONMENT="test" \
-e BITOPS_ENVIRONMENT="test" \
-v $(pwd):/opt/bitops_deployment \
-v $BITOPS_HOME:/opt/bitops \
bitovi/bitops:latest
Expand All @@ -92,7 +93,7 @@ bitovi/bitops:latest
### Using Declarative Infrastructure to Deploy an EKS Cluster and Helm Chart
* [Using Declarative Infrastructure to Deploy an EKS Cluster and Helm Chart](https://www.bitovi.com/blog/eks-helm-bitops)

### Bitops and Terraform
### BitOps and Terraform
* [BitOps + Terraform](https://www.bitovi.com/blog/bitops-terraform)

### Combine Terraform and Ansible to provision and configure a webserver
Expand Down
7 changes: 3 additions & 4 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,20 @@ See [configuration](configuration-base.md)

## Run BitOps

BitOps is packaged as a docker image and is available on [dockerhub](https://hub.docker.com/r/bitovi/bitops).
BitOps is packaged as a docker image and is available on [docker hub](https://hub.docker.com/r/bitovi/bitops).

```
docker pull bitovi/bitops
cd $YOUR_OPERATIONS_REPO
docker run bitovi/bitops -v $(pwd):/opt/bitops_deployment
```
If you need specific version of BitOps, please check the [BitOps versioning](versioning.md).
If you need a specific version of BitOps, please check the [BitOps versioning](versioning.md).

## Supported Tools
* [Provision infrastructure with CloudFormation](tool-configuration/configuration-cloudformation.md)
* [Provision infrastructure with Terraform](tool-configuration/configuration-terraform.md)
* [Configure infrastructure with Ansible](tool-configuration/configuration-ansible.md)
* [Deploy to kubernetes with Helm](tool-configuration/configuration-helm.md)
* [Deploy to Kubernetes with Helm](tool-configuration/configuration-helm.md)

## Supported Cloud Providers

Expand All @@ -51,4 +51,3 @@ If you need specific version of BitOps, please check the [BitOps versioning](ver
## Guides and Other Resources

BitOps already has several guides demonstrating deploying a web server or a pre-configured application using different combinations of the available BitOps tools. If you're looking for a quick start, check out some of the options available in the [Examples](examples.md) section.

28 changes: 13 additions & 15 deletions docs/lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,55 +3,53 @@
## Lifecycle hooks
Within each tool directory, you can optionally have a `bitops.before-deploy.d/` and/or a `bitops.after-deploy.d/`. If any shell scripts exist within these directories, BitOps will execute them in alphanumeric order.

This is a useful way to extend the functionality of BitOps. A popular usecase we've seen is loading secrets or dynamically editing `bitops.config.yml`
This is a useful way to extend the functionality of BitOps. A popular use case we've seen is loading secrets, preparing the environment or dynamically editing `bitops.config.yaml`.

## Detailed Execution Flow

![lifecycle diagram](assets/images/lifecycle.png)

A single run of BitOps will
A single run of BitOps will:

1. Copy the contents of `/opt/bitops_deployment` to a temporary working directory
2. Attempt to setup a cloud provider
3. If a `terraform/` directory exists within the selected environment
3. If a `terraform/` directory exists within the selected environment:
* Run any `bitops.before-deploy.d/*.sh` scripts
* Load `bitops.config.yml` and set environment
* Load `bitops.config.yaml` and set environment
* Merge contents with [Default environment](default-environment.md) - [TODO](https://github.com/bitovi/bitops/issues/18)
* Select terraform version
* Run `terraform init`
* Select `terraform workspace`
* Run `terraform plan`
* Run `terraform apply` or `terraform destroy`
* Run any `bitops.after-deploy.d/*.sh` scripts
4. If a `ansible/` directory exists within the selected environment
4. If a `ansible/` directory exists within the selected environment:
* Run any `bitops.before-deploy.d/*.sh` scripts
* Load `bitops.config.yml` and set environment
* Load `bitops.config.yaml` and set environment
* Merge contents with [Default environment](default-environment.md) - [TODO](https://github.com/bitovi/bitops/issues/18)
* Run `ansible-playbook $playbook` for each `*.yaml` or `*.yml` file in `$env/ansible/`
* Run any `bitops.after-deploy.d/*.sh` scripts
4. If a `helm/` directory exists within the selected environment
4. If a `helm/` directory exists within the selected environment:
* Run the following for `$env/helm/$ENVIRONMENT_HELM_SUBDIRECTORY/` or for all charts in `$env/helm/`
* Run any `bitops.before-deploy.d/*.sh` scripts
* Load `bitops.config.yml` and set environment
* Load `bitops.config.yaml` and set environment
* Merge contents with [Default environment](default-environment.md)
* Use `$KUBE_CONFIG_PATH` if defined, if not use aws cli to build .kubeconfig
* Use `$KUBE_CONFIG_PATH` if defined, if not use AWS CLI to build `.kubeconfig`
* Gather all values files - TODO document
* Run `helm dep up`
* Run `helm upgrade` or `helm install`
* Run `helm rollback` on failure
* Run any `bitops.after-deploy.d/*.sh` scripts
* TODO `helm_install_external_charts` and `helm_install_charts_from_s3` never run!
4. If a `cloudformation/` directory exists within the selected environment
4. If a `cloudformation/` directory exists within the selected environment:
* Run any `bitops.before-deploy.d/*.sh` scripts
* Load `bitops.config.yml` and set environment
* Load `bitops.config.yaml` and set environment
* Merge contents with [Default environment](default-environment.md) - [TODO](https://github.com/bitovi/bitops/issues/18)
* Run cfn template validation
* Create or delete cfn stack. Wait for completion
* Run any `bitops.after-deploy.d/*.sh` scripts

### Environment Variables
Plugins can export environment when a value is specified in a ops_repo level `bitops.config.yaml`. These environment variables are prefixed with `BITOPS` and their plugin name.
Plugins can export the environment when a value is specified in an ops_repo level `bitops.config.yaml`. These environment variables are prefixed with `BITOPS_` and their plugin name.

So for example, if the terraform plugin exported the environment variable BUTTER_FLAG, it would be accessible in the lifecycle hooks by referencing;

`BITOPS_TERRAFORM_BUTTER_FLAG`
So for example, if the terraform plugin exported the environment variable `BUTTER_FLAG`, it would be accessible in the lifecycle hooks by referencing `BITOPS_TERRAFORM_BUTTER_FLAG`.
Loading

0 comments on commit 947b4d2

Please sign in to comment.