Event-Based Mirroring: Listen for ECS Service Events

[chelma]

Description

Per design discussion in (#35), we want to begin listening for and responding to AWS Service-emitted EventBridge Events to more swiftly capture new user traffic. This task is to listen for ECS Containers starting up/spinning down and automatically create mirroring configuration for them without having to take manual action or wait for a scheduled scan.

This task will likely entail:

• Update the existing Subnet-based configuration Lambda (see Event-Based Mirroring: Scheduled Scans #36) to just detect new/removed ENIs and kick off additional Lambdas to add/remove mirroring resources rather than manipulate the mirroring resources itself.
• Create new Rules to listen for per-ENI mirroring create/remove requests and new Lambdas to action them. These new Lambdas will probably just contain the code that previously existed in the Subnet-scan Lambda.
• Create new Rules to listen for ECS Container start/stop events on the AWS Account/Region's default Bus and kick off the appropriate per-ENI Lambda to action it.

Some design comments:

• After this task is completely, we'll have two levels of Lambdas:
  • A Subnet-level lambda that scans for new/removed ENIs in a specified Subnet, which is invoked on a schedule and when the user invokes add-vpc or remove-vpc on the CLI. It offloads the responsibility of manipulating mirroring resources to...
  • Two ENI-level Lambdas for creating/removing per-ENI mirroring configuration (separate Lambda for each case), which will be invoked directly when an ECS Service event is detected, and as a second-order action when the Subnet-level "scan" lambda detects changes in ENIs.
• We tackle ECS containers first because we already have a CLI command (deploy-demo-traffic) that spins up example ECS Fargate containers, which makes it easier to test than if we tackled EC2 Events first.

Acceptance Criteria

• Demonstrate that mirroring configuration is correctly created/destroyed when ECS Containers are added/removed from the User VPC WITHOUT having to wait for a scheduled scan.

[chelma]

Decided to tackle this before (#36) as I think it will be quicker to implement and easier to test with this ordering.

[chelma]

PR Posted for the first part of the work (#42). This is mostly some setup work and library code that I tested by switching the add-vpc and remove-vpc CLI commands over to kicking off events instead of directly creating per-ENI mirroring resources themselves.

We need a README section that explains how this stuff works; will add that in my next PR to wrap this task up.

[chelma]

Readme updated for events (#48)

[chelma]

Since we did this before the Scheduled Scans task (#36), the task description and acceptance criteria do no quite map, but the criteria have been fulfilled.

Per the code updates, we can actually service changes to both ECS-on-EC2 and EC2 resources by just looking at EC2 events. If we also look at Fargate events, we can service all of these use-cases. We're now doing that with these code updates.

Resolving.