Design mechanism for auto-creation of per-ENI mirroring

[chelma]

Description

Currently, the only way to update which ENIs are monitored in a particular user-VPC is invoke remove-vpc & add-vpc to tear down capture and stand it back up, a process with currently takes ~10 minutes. This is obviously not the desired experience.

This task is to investigate options for more nimble, ideally automatic, creation and destruction of per-ENI mirroring configuration and propose a initial solution to the problem. The proposal should ideally address the following topics:

• Automated creation/destruction of per-ENI configuration in each user-VPC
• A way for users to initiate this process themselves if they wish
• How the system will behave for longer-lived (EC2 instances) and shorter lived (Lambda containers) resources
• How the system will handle multiple, concurrent and/or conflicting instructions

This task will NOT address changes in the network topography of the user's VPC (e.g. adding/removing subnets); that will be addressed with the refresh-cluster command (see: #32)

Acceptance Criteria

• Proposal design, written up in GitHub

[chelma]

Taking a look at things, AWS Event Bridge [1] seems like a good option to explore further. I'm thinking we'll have one or more Lambda functions that are capable of setting up/tearing down the per-ENI resources. We'll probably schedule them to fire on a regular cadence as a backstop, and then use Event Bridge to catch things like EC2 ASG and ECS scale up/down events and react sooner than backstop would fire. LOTS of AWS Services fire these events [2].

[1] https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-what-is.html
[2] https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-service-event-list.html

[chelma]

Design proposal posted: #35

[chelma]

Looks like we're aligned on the design proposal. Created follow-up tasks:

• Event-Based Mirroring: Scheduled Scans #36
• Event-Based Mirroring: Listen for ECS Service Events #37
• Event-Based Mirroring: Listen for EC2 Service Events #38
• Event-Based Mirroring: Automated per-Subnet Configuration #39

[chelma]

Task completed; resolving.