Skip to content

Commit

Permalink
Merge pull request #180 from GlueOps/feature/fluentbit-s3-credentials
Browse files Browse the repository at this point in the history
Feature/fluentbit s3 credentials
  • Loading branch information
venkatamutyala authored May 20, 2024
2 parents c2e5a08 + de3d2dd commit d17bacc
Show file tree
Hide file tree
Showing 5 changed files with 67 additions and 7 deletions.
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,12 @@ No requirements.

| Name | Source | Version |
|------|--------|---------|
| <a name="module_argocd_helm_values"></a> [argocd\_helm\_values](#module\_argocd\_helm\_values) | git::https://github.com/GlueOps/docs-argocd.git | v0.12.1 |
| <a name="module_argocd_helm_values"></a> [argocd\_helm\_values](#module\_argocd\_helm\_values) | git::https://github.com/GlueOps/docs-argocd.git | v0.13.0 |
| <a name="module_captain_repository"></a> [captain\_repository](#module\_captain\_repository) | ./modules/github-captain-repository/0.1.0 | n/a |
| <a name="module_captain_repository_files"></a> [captain\_repository\_files](#module\_captain\_repository\_files) | ./modules/github-captain-repository-files/0.1.0 | n/a |
| <a name="module_common_s3"></a> [common\_s3](#module\_common\_s3) | ./modules/multy-s3-bucket/0.1.0 | n/a |
| <a name="module_dnssec_key"></a> [dnssec\_key](#module\_dnssec\_key) | git::https://github.com/GlueOps/terraform-module-cloud-aws-dnssec-kms-key.git | v0.3.0 |
| <a name="module_glueops_platform_helm_values"></a> [glueops\_platform\_helm\_values](#module\_glueops\_platform\_helm\_values) | git::https://github.com/GlueOps/platform-helm-chart-platform.git | v0.42.0 |
| <a name="module_glueops_platform_helm_values"></a> [glueops\_platform\_helm\_values](#module\_glueops\_platform\_helm\_values) | git::https://github.com/GlueOps/platform-helm-chart-platform.git | v0.43.0-rc10 |
| <a name="module_loki_s3"></a> [loki\_s3](#module\_loki\_s3) | ./modules/multy-s3-bucket/0.1.0 | n/a |
| <a name="module_opsgenie_teams"></a> [opsgenie\_teams](#module\_opsgenie\_teams) | ./modules/opsgenie/0.1.0 | n/a |
| <a name="module_tenant_readmes"></a> [tenant\_readmes](#module\_tenant\_readmes) | ./modules/tenant-readme/0.1.0 | n/a |
Expand All @@ -59,12 +59,14 @@ No requirements.
|------|------|
| [aws_iam_access_key.certmanager](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_access_key.externaldns](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_access_key.fluentbit_log_exporter_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_access_key.loki_log_exporter_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_access_key.loki_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_access_key.tls_cert_backup_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_access_key.tls_cert_restore_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_access_key.vault_init_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_access_key.vault_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_policy.fluentbit_logs_exporter_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.loki_logs_exporter_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.loki_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.route53](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
Expand All @@ -74,6 +76,7 @@ No requirements.
| [aws_iam_policy.vault_s3_backup](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_user.certmanager](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
| [aws_iam_user.externaldns](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
| [aws_iam_user.fluentbit_log_exporter_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
| [aws_iam_user.loki_log_exporter_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
| [aws_iam_user.loki_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
| [aws_iam_user.tls_cert_backup_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
Expand All @@ -82,6 +85,7 @@ No requirements.
| [aws_iam_user.vault_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
| [aws_iam_user_policy_attachment.certmanager](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) | resource |
| [aws_iam_user_policy_attachment.externaldns](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) | resource |
| [aws_iam_user_policy_attachment.fluentbit_log_exporter_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) | resource |
| [aws_iam_user_policy_attachment.loki_log_exporter_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) | resource |
| [aws_iam_user_policy_attachment.loki_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) | resource |
| [aws_iam_user_policy_attachment.tls_cert_backup_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) | resource |
Expand Down
10 changes: 6 additions & 4 deletions generate-helm-values.tf
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,14 @@ resource "random_password" "grafana_admin_secret" {
}

locals {
vault_access_tokens_s3_key = "hashicorp-vault-init/vault_access.json"
tls_cert_backup_s3_key_prefix = "tls-cert-backups"
vault_access_tokens_s3_key = "hashicorp-vault-init/vault_access.json"
tls_cert_backup_s3_key_prefix = "tls-cert-backups"
tls_cert_restore_exclude_namespaces = "kube-system"
}

module "glueops_platform_helm_values" {
for_each = local.environment_map
source = "git::https://github.com/GlueOps/platform-helm-chart-platform.git?ref=v0.42.0"
source = "git::https://github.com/GlueOps/platform-helm-chart-platform.git?ref=v0.43.0-rc10"
captain_repo_b64encoded_private_deploy_key = base64encode(module.captain_repository[each.value.environment_name].private_deploy_key)
captain_repo_ssh_clone_url = module.captain_repository[each.value.environment_name].ssh_clone_url
this_is_development = var.this_is_development
Expand All @@ -58,6 +58,8 @@ module "glueops_platform_helm_values" {
loki_aws_secret_key = aws_iam_access_key.loki_s3[each.value.environment_name].secret
loki_exporter_aws_access_key = aws_iam_access_key.loki_log_exporter_s3[each.value.environment_name].id
loki_exporter_aws_secret_key = aws_iam_access_key.loki_log_exporter_s3[each.value.environment_name].secret
fluentbit_exporter_aws_access_key = aws_iam_access_key.fluentbit_log_exporter_s3[each.value.environment_name].id
fluentbit_exporter_aws_secret_key = aws_iam_access_key.fluentbit_log_exporter_s3[each.value.environment_name].secret
certmanager_aws_access_key = aws_iam_access_key.certmanager[each.value.environment_name].id
certmanager_aws_secret_key = aws_iam_access_key.certmanager[each.value.environment_name].secret
externaldns_aws_access_key = aws_iam_access_key.externaldns[each.value.environment_name].id
Expand Down Expand Up @@ -104,7 +106,7 @@ resource "aws_s3_object" "platform_helm_values" {

module "argocd_helm_values" {
for_each = local.environment_map
source = "git::https://github.com/GlueOps/docs-argocd.git?ref=v0.12.1"
source = "git::https://github.com/GlueOps/docs-argocd.git?ref=v0.13.0"
tenant_key = var.tenant_key
cluster_environment = each.value.environment_name
client_secret = random_password.dex_argocd_client_secret[each.value.environment_name].result
Expand Down
34 changes: 34 additions & 0 deletions iam-policy-fluentbit-xptr-s3.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
resource "aws_iam_policy" "fluentbit_logs_exporter_s3" {
provider = aws.clientaccount
for_each = aws_route53_zone.clusters
name = "fluentbit-xptr-s3-${aws_route53_zone.clusters[each.key].name}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:List*"
],
"Resource": [
"${module.common_s3.primary_s3_bucket_arn}/${aws_route53_zone.clusters[each.key].name}/fluentbit_exported_logs/*",
"${module.common_s3.replica_s3_bucket_arn}/${aws_route53_zone.clusters[each.key].name}/fluentbit_exported_logs/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"${module.common_s3.primary_s3_bucket_arn}",
"${module.common_s3.replica_s3_bucket_arn}"
]
}
]
}
EOF
}
20 changes: 20 additions & 0 deletions iam-user-fluentbit-xptr-s3.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
resource "aws_iam_user" "fluentbit_log_exporter_s3" {
provider = aws.clientaccount
for_each = aws_route53_zone.clusters
name = "fluentbit-xptr-s3-${aws_route53_zone.clusters[each.key].name}"
}

resource "aws_iam_user_policy_attachment" "fluentbit_log_exporter_s3" {
provider = aws.clientaccount
for_each = aws_iam_user.fluentbit_log_exporter_s3
user = each.value.name
policy_arn = aws_iam_policy.fluentbit_logs_exporter_s3[each.key].arn
depends_on = [aws_iam_user.fluentbit_log_exporter_s3]
}

resource "aws_iam_access_key" "fluentbit_log_exporter_s3" {
for_each = aws_iam_user.fluentbit_log_exporter_s3
provider = aws.clientaccount
user = each.value.name
depends_on = [aws_iam_user.fluentbit_log_exporter_s3]
}
2 changes: 1 addition & 1 deletion modules/tenant-readme/0.1.0/readme.tf
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ locals {
codespace_version = "v0.42.0"
argocd_crd_version = var.argocd_app_version
argocd_helm_chart_version = "5.54.0"
glueops_platform_version = "v0.42.0" # this also needs to be updated in the module.glueops_platform_helm_values // generate-helm-values.tf
glueops_platform_version = "v0.43.0-rc10" # this also needs to be updated in the module.glueops_platform_helm_values // generate-helm-values.tf
tools_version = "v0.10.0"
}

Expand Down

0 comments on commit d17bacc

Please sign in to comment.