Skip to content

Latest commit

 

History

History
184 lines (131 loc) · 7.59 KB

Readme.md

File metadata and controls

184 lines (131 loc) · 7.59 KB
Raw

cidr-house-rules

Build Status

A lightweight API and collection system to centralize important AWS resource information across multiple accounts in near-realtime

cidr-house-rules

Collection subsystem

Centralized AWS Lambda functions assume role on monitored AWS accounts to collect the following:

Items collected into Dynamodb will expire if no longer found (default is 48 hours). TTLs are controlled via environment variables passed to each import function. TTL time is expressed in seconds. It is a calculation of current time + TTL. Each import related DynamoDB table leverages TTL for object expiration

Collection subsystem runner process

The above noted cidr-house-rules collection functions are triggered by a runner function which invokes the necessary number of import functions based upon the number of AWS accounts managed and number of regions provided by AWS. The runner process allows for cidr-house-rules to scale given any number of AWS accounts to collect information from.

Import functions invokation visual

import-function-invoke

API interface

An API interface is provided to expose collected data for consumption. Example usage is through Terraform's http data source

  • retrieve all NAT Gateways for a given AWS account (team)
  • retrieve all EIPs for a given AWS account (team)
  • lookup a given CIDR block, i.e. 10.0.16.0/16 for potential VPC peering conflicts

Terraform modules that use cidr-house-rules for dynamic data

Deployment

  1. Requires serverless-aws-documentation plugin
  2. Pick an AWS account to deploy cidr-house-rules to.
  3. On remote accounts applying the supporting terraform see link below to generate role access
  4. With roles generated on remote accounts you can now move serverless.yml.example to serverless.yml and add your target account roles
  5. Run serverless deploy --stage prod

Onboarding new AWS accounts

  1. Apply the following Terraform and obtain outputs
provider "aws" {
  region = "us-west-2"
}

module "cidr-house-rules-role" {
  cidr_house_rules_account_number = "123456770"
  serverless_stage                = "prod"
  serverless_region               = "us-west-2"
  source                          = "git::ssh://git@github.com/trulia/cidr-house-rules-role.git?ref=v0.0.1"
}

output "account_alias" {
  value = "${module.cidr-house-rules-role.account_alias}"
}

output "account_id" {
  value = "${module.cidr-house-rules-role.account_id}"
}

output "cidr-house-rules-role" {
  value = "${module.cidr-house-rules-role.cidr-house-rules-role}"
}
  1. With the account_alias, account_id to app
curl \
  --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
  https://yourapigateway-endpoint-generated-by-serverless.com/prod/add_account?team=trucomms?account=35682931234
  1. Add the role for Lambda functions to use in serverless.yml
Under this section add the new role, there should be a list of them already
iamRoleStatements:
  - Effect: Allow
    Action: sts:AssumeRole
    Resource:
      arn:aws:iam::<remote_aws_account_number_here>:role/role_cidr_house
  1. Run deployment job in Jenkins noted in the "Deployment" section

Example API calls

Obtain NAT gateways for platform, webteam and dataeng teams
curl \
  --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
   https://yourapigateway-endpoint-generated-by-serverless.com/prod/get_nat_gateways_for_team?platform&webteam&dataeng

# Or just one team
curl \
  --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
  https://yourapigateway-endpoint-generated-by-serverless.com/prod/get_nat_gateways_for_team?team=platform
Obtain NAT gateways for all monitored accounts
curl \
  --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
   https://yourapigateway-endpoint-generated-by-serverless.com/prod/get_nat_gateways_for_all
Obtain number of result pages for NAT gatways, useful for Terraform to count out resources on
# Default is 50 results per page
curl --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" 'https://yourapigateway-endpoint-generated-by-serverless.com/prod/get_number_of_nat_gateway_pages'
# Request number of results per page to be 10, and return total number of pages
curl --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" 'https://yourapigateway-endpoint-generated-by-serverless.com/prod/get_number_of_nat_gateway_pages?results_per_page=10'
Obtain paged results of Nat gateways results
curl --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" 'https://yourapigateway-endpoint-generated-by-serverless.com/prod/get_nat_gateways_for_all?results_per_page=10&page=4'
Check for a VPC CIDR conflict
curl \
  --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
  https://yourapigateway-endpoint-generated-by-serverless.com/dev/check_conflict?cidr=10.17.0.0/16
Obtain all ELBs, ALBs, NLBs across all accounts
curl \
--header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
https://yourapigateway-endpoint-generated-by-serverless.com/dev/get_elbs_for_all
Obtain all PrivateLink endpoint services across all accounts
curl \
--header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
https://yourapigateway-endpoint-generated-by-serverless.com/dev/get_service_endpoints_for_all
Obtain a specific PrivateLink endpoint service based upon the "Name" tag of an NLB associated with it. (AWS presently doesn't allow tagging of PrivateLink endpoint services, so next best option is to use tags of NLB associated with PrivateLink)
curl \
--header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
https://yourapigateway-endpoint-generated-by-serverless.com/dev/get_service_endpoints_for_nlb?nlb=my-nlb
Add a new account
curl \
  --header "X-Api-Key: <GET_KEY_FROM_AWS_API_GATEWAY>" \
  https://yourapigateway-endpoint-generated-by-serverless.com/dev/add_account?team=my_aws_account_alias_here&account=35682931234
Supporting Terraform

cird-house-rules-role - use this Terraform on your accounts you would like your cidr-house-rules deployment to have access to. The outputs on this Terraform can then be used in your serverless.yml. The account number of the remote account can then be added using the add_account API endpoint.