Skip to content

Commit

Permalink
datasources
Browse files Browse the repository at this point in the history
  • Loading branch information
JamesWoolfenden committed Feb 3, 2023
1 parent c83dae7 commit 125bced
Show file tree
Hide file tree
Showing 41 changed files with 261 additions and 70 deletions.
7 changes: 6 additions & 1 deletion .markdownlint.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
{
"MD013": false
"MD013": false,
"MD033": {
"allowed_elements": [
"cloud"
]
}
}
8 changes: 4 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ repos:
- id: detect-aws-credentials
- id: detect-private-key
- repo: https://github.com/Lucas-C/pre-commit-hooks
rev: v1.3.1
rev: v1.4.2
hooks:
- id: forbid-tabs
exclude_types: [python, javascript, dtd, markdown, makefile, xml]
Expand All @@ -28,7 +28,7 @@ repos:
hooks:
- id: shell-lint
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.32.2
rev: v0.33.0
hooks:
- id: markdownlint
exclude: src/testdata|testdata
Expand All @@ -38,13 +38,13 @@ repos:
- id: terraform-fmt
language_version: python3.9
- repo: https://github.com/gruntwork-io/pre-commit
rev: v0.1.17
rev: v0.1.18
hooks:
- id: gofmt
- id: goimports
- id: golint
- repo: https://github.com/golangci/golangci-lint
rev: v1.50.1
rev: v1.51.0
hooks:
- id: golangci-lint
- repo: https://github.com/syntaqx/git-hooks
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -69,4 +69,4 @@ psbump:

update:
go get -u
go mod tidy
go mod tidy
26 changes: 13 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ scoop bucket add iac https://github.com/JamesWoolfenden/scoop.git

Then you can install a tool:

```
```bash
scoop install pike
```

Expand Down Expand Up @@ -539,29 +539,29 @@ e.g. *aws_security_group.json*
### How

Datasources are the easiest to start with, I have a script (resource.ps1 - add pwsh with **brew install --cask powershell**)
that creates a blank mapping file and tf
that creates a blank mapping file and tf
resource, but you've seen the example json file - make one without any entries.
You also need to create a minimal resource/datasource, that you are trying to figure out the permissions for, and place it in the correct dir
You also need to create a minimal resource/datasource, that you are trying to figure out the permissions for, and place it in the correct dir
e.g../terraform/aws, I have a script for making a profile for the profile in the role directory.
You can then tf using the empty role against the resource/datasource with no permissions.
You can then tf using the empty role against the resource/datasource with no permissions.
The debug output from the tf run will help you figure out the permissions you need to add to your basic role.
You then update your "basic" role.
You then update your "basic" role.

Issues?
The providers don't always tell you want you need to add,
you will need to check the IAM docs and the online IAM policymakers.
Not all resource are as easy as others, anything that make/scripts CF internally.
Issues?
The providers don't always tell you want you need to add,
you will need to check the IAM docs and the online IAM policymakers.
Not all resource are as easy as others, anything that make/scripts CF internally.
Some roles require *Passrole* and *CreateLinkedRole* but won't say so. Trail and error

#### What about "attributes" ?

Some cloud providers require extra permissions depending on the attributes you add, this is how this is handled.
Build out your tf resources to cover all reasonable scenarios.
Some cloud providers require extra permissions depending on the attributes you add, this is how this is handled.
Build out your tf resources to cover all reasonable scenarios.

#### Eventual consistency

Some cloud providers follow this model which means your test IAM role will take time after you change it to be
changed, how long? This seems to vary on time of day and the resource. Whilst other providers like
Some cloud providers follow this model which means your test IAM role will take time after you change it to be
changed, how long? This seems to vary on time of day and the resource. Whilst other providers like
Azure just take a long time for the TF to change.

### Add Import mapping file
Expand Down
8 changes: 8 additions & 0 deletions src/aws_datasource.go
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,14 @@ func GetAWSDataPermissions(result ResourceV2) ([]string, error) {
"aws_lb": dataAwsLb,
"aws_secretsmanager_secret": dataAwsSecretsmanagerSecret,
"aws_secretsmanager_secret_version": dataAwsSecretsmanagerSecretVersion,
"aws_sesv2_dedicated_ip_pool": dataAwsSesv2DedicatedIPPool,
"aws_sqs_queues": dataAwsSqsQueues,
"aws_vpc_ipam_pools": dataAwsVpcIpamPools,
"aws_auditmanager_control": dataAwsAuditmanagerControl,
"aws_auditmanager_framework": dataAwsAuditmanagerFramework,
"aws_connect_instance_storage_config": dataAwsConnectInstanceStorageConfig,
"aws_controltower_controls": dataAwsControltowerControls,
"aws_db_instances": dataAwsDbInstances,
}

var Permissions []string
Expand Down
24 changes: 24 additions & 0 deletions src/files_datasource.go
Original file line number Diff line number Diff line change
Expand Up @@ -342,3 +342,27 @@ var dataAwsSecretsmanagerSecret []byte

//go:embed mapping/aws/data/secretsmanager/aws_secretsmanager_secret_version.json
var dataAwsSecretsmanagerSecretVersion []byte

//go:embed mapping/aws/data/ses/aws_sesv2_dedicated_ip_pool.json
var dataAwsSesv2DedicatedIPPool []byte

//go:embed mapping/aws/data/sqs/aws_sqs_queues.json
var dataAwsSqsQueues []byte

//go:embed mapping/aws/data/ec2/aws_vpc_ipam_pools.json
var dataAwsVpcIpamPools []byte

//go:embed mapping/aws/data/auditmanager/aws_auditmanager_control.json
var dataAwsAuditmanagerControl []byte

//go:embed mapping/aws/data/auditmanager/aws_auditmanager_framework.json
var dataAwsAuditmanagerFramework []byte

//go:embed mapping/aws/data/connect/aws_connect_instance_storage_config.json
var dataAwsConnectInstanceStorageConfig []byte

//go:embed mapping/aws/data/controltower/aws_controltower_controls.json
var dataAwsControltowerControls []byte

//go:embed mapping/aws/data/rds/aws_db_instances.json
var dataAwsDbInstances []byte
13 changes: 13 additions & 0 deletions src/mapping/aws/data/auditmanager/aws_auditmanager_control.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"apply": [],
"attributes": {
"tags": []
},
"destroy": [],
"modify": [],
"plan": [
"auditmanager:ListControls"
]
}
]
13 changes: 13 additions & 0 deletions src/mapping/aws/data/auditmanager/aws_auditmanager_framework.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"apply": [],
"attributes": {
"tags": []
},
"destroy": [],
"modify": [],
"plan": [
"auditmanager:ListAssessmentFrameworks"
]
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"apply": [],
"attributes": {
"tags": []
},
"destroy": [],
"modify": [],
"plan": [
"connect:DescribeInstanceStorageConfig"
]
}
]
13 changes: 13 additions & 0 deletions src/mapping/aws/data/controltower/aws_controltower_controls.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"apply": [],
"attributes": {
"tags": []
},
"destroy": [],
"modify": [],
"plan": [
"controltower:ListEnabledControls"
]
}
]
2 changes: 1 addition & 1 deletion src/mapping/aws/data/ec2/aws_vpc_ipam_pool_cidrs.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"destroy": [],
"modify": [],
"plan": [
"ec2:DescribeIpamPools"
"ec2:GetIpamPoolCidrs"
]
}
]
13 changes: 13 additions & 0 deletions src/mapping/aws/data/ec2/aws_vpc_ipam_pools.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"apply": [],
"attributes": {
"tags": []
},
"destroy": [],
"modify": [],
"plan": [
"ec2:DescribeIpamPools"
]
}
]
13 changes: 13 additions & 0 deletions src/mapping/aws/data/rds/aws_db_instances.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"apply": [],
"attributes": {
"tags": []
},
"destroy": [],
"modify": [],
"plan": [
"rds:DescribeDBInstances"
]
}
]
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
},
"destroy": [],
"modify": [],
"plan": [ "servicequotas:ListServices"]
"plan": [
"servicequotas:ListServices"
]
}
]
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
},
"destroy": [],
"modify": [],
"plan": [ "servicequotas:ListServices"]
"plan": [
"servicequotas:ListServices"
]
}
]
13 changes: 13 additions & 0 deletions src/mapping/aws/data/ses/aws_sesv2_dedicated_ip_pool.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"apply": [],
"attributes": {
"tags": []
},
"destroy": [],
"modify": [],
"plan": [
"ses:GetDedicatedIpPool"
]
}
]
13 changes: 13 additions & 0 deletions src/mapping/aws/data/sqs/aws_sqs_queues.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
[
{
"apply": [],
"attributes": {
"tags": []
},
"destroy": [],
"modify": [],
"plan": [
"sqs:ListQueues"
]
}
]
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,20 @@
"elasticache:ListTagsForResource",
"elasticache:CreateUserGroup",
"elasticache:DescribeUserGroups",
"elasticache:DeleteUserGroup"],
"elasticache:DeleteUserGroup"
],
"attributes": {
"tags": [
"elasticache:AddTagsToResource",
"elasticache:RemoveTagsFromResource"
]
},
"destroy": [
"elasticache:DeleteUserGroup"],
"elasticache:DeleteUserGroup"
],
"modify": [
"elasticache:ModifyUserGroup"],
"elasticache:ModifyUserGroup"
],
"plan": []
}
]
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
[
{
"apply": [
"elasticloadbalancing:CreateRule",
"elasticloadbalancing:SetRulePriorities"
],
"attributes": {
"tags": [
"elasticloadbalancing:AddTags",
"elasticloadbalancing:RemoveTags",
"elasticloadbalancing:DescribeTags"
]
},
"destroy": [
"elasticloadbalancing:DeleteRule"
],
"modify": [
"elasticloadbalancing:ModifyRule"
],
"plan": [
"elasticloadbalancing:DescribeRules"
]
}
]
[
{
"apply": [
"elasticloadbalancing:CreateRule",
"elasticloadbalancing:SetRulePriorities"
],
"attributes": {
"tags": [
"elasticloadbalancing:AddTags",
"elasticloadbalancing:RemoveTags",
"elasticloadbalancing:DescribeTags"
]
},
"destroy": [
"elasticloadbalancing:DeleteRule"
],
"modify": [
"elasticloadbalancing:ModifyRule"
],
"plan": [
"elasticloadbalancing:DescribeRules"
]
}
]
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
"apply": [
"iam:AddUserToGroup",
"iam:ListGroupsForUser",
"iam:RemoveUserFromGroup"],
"iam:RemoveUserFromGroup"
],
"attributes": {
"tags": []
},
"destroy": [
"iam:RemoveUserFromGroup"],
"iam:RemoveUserFromGroup"
],
"modify": [],
"plan": []
}
Expand Down
4 changes: 3 additions & 1 deletion src/mapping/aws/resource/lambda/aws_lambda_invocation.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
[
{
"apply": ["lambda:InvokeAsync"],
"apply": [
"lambda:InvokeAsync"
],
"attributes": {
"tags": []
},
Expand Down
3 changes: 2 additions & 1 deletion src/mapping/aws/resource/s3/aws_s3_bucket_inventory.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
{
"apply": [
"s3:GetInventoryConfiguration",
"s3:PutInventoryConfiguration"],
"s3:PutInventoryConfiguration"
],
"attributes": {
"tags": []
},
Expand Down
Loading

0 comments on commit 125bced

Please sign in to comment.