Apply clusterSecurityGroupTags to the EKS created primary Cluster Security Group

[sanketg86]

What happened?

When I try to use clusterSecurityGroupTags parameter in eks.cluster package to create cluster and assign some more tags to Cluster security group, it assigned to additional security group.

Example

new eks.Cluster('eks-cluster',
      {
        name: 'eks-cluster',
        vpcId: vpc.id,
        serviceRole: eksRole,
        encryptionConfigKeyArn: kmsKey.arn,
        instanceRole: workerNodeRole,
        instanceProfileName: nodeInstanceProfile.name,
        skipDefaultNodeGroup: true,
        createOidcProvider: true,
        endpointPrivateAccess: true,
        endpointPublicAccess: true,
        useDefaultVpcCni: true,
        clusterSecurityGroupTags: { // <--- These listed tags must goes to actual cluster security group
          Name: 'eks-cluster',
          'karpenter.sh/discovery': 'eks-cluster' 
        },
        version: 1.29,
}
)

Output of pulumi about

CLI          
Version      3.119.0
Go Version   go1.22.3
Go Compiler  gc

Host     
OS       ubuntu
Version  22.04
Arch     x86_64

Plugins
KIND      NAME        VERSION
resource  aws               6.39.1
resource  eks               2.7.0
resource  kubernetes  4.13.1

Dependencies:
NAME                VERSION
@pulumi/eks         2.7.0
@pulumi/kubernetes  4.13.1
@pulumi/pulumi      3.119.0
@types/node         20.14.2
ts-node             10.9.2
yaml                2.4.5
@pulumi/aws         6.39.1

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[rquitales]

Hi @sanketg86 - I'm not familiar with karpenter and how it does discovery with security group tags, but Is it insufficient for this tags to be within a separate additional security group attached to the EKS cluster? This is how our EKS provider is architected currently since the default security group is created automatically by AWS whenever a new EKS cluster is provision to enable communication between the worker nodes and K8s control plane. There isn't a straightforward way to modify this security group during the pulumi up lifecycle afaict. My understanding is also that additional security groups should behave the same as amalgamating all config within the one created by AWS.

[sanketg86]

as EKS cluster security group assigned to AWS manage node and if some workload running on AWS managed nodegroup which assigned EKS cluster security group and some workload running on Karpenter managed nodegroup with different security group, they can not communicate with each other.

so I would like to assign same security group to Karpenter nodes which assign to AWS managed nodegroup for that i have to add Karpenter discovery tag to cluster SG.

[rquitales]

Thanks for the additional info here. I'll confer with the team to see if there's a way to resolve this.
cc @flostadler

[flostadler]

@sanketg86 the primary cluster security group is auto created by EKS itself and not within pulumi. You could tag it like this:

const primaryClusterSecurityGroup = cluster.eksCluster.vpcConfig.clusterSecurityGroupId;
new aws.ec2.Tag("exampletagResourceResourceFromEc2tag", {
    key: "my-tag",
    resourceId: primaryClusterSecurityGroup,
    value: "my-value",
});

As an enhancement we could do this for users automatically unless they opt out of it.