Skip to content
This repository has been archived by the owner on May 31, 2024. It is now read-only.

AGC not working with enforced S3 encryption #574

Open
ArlindNocaj opened this issue Nov 23, 2022 · 0 comments
Open

AGC not working with enforced S3 encryption #574

ArlindNocaj opened this issue Nov 23, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@ArlindNocaj
Copy link

ArlindNocaj commented Nov 23, 2022 •
edited
Loading

Describe the Bug
The only way how AWS allows to enforce encryption on S3 is using SCP: https://aws.amazon.com/blogs/security/how-to-prevent-uploads-of-unencrypted-objects-to-amazon-s3/

AGC does not work when S3 encryption is being enforced through SCP.
Most larger enterprises use such a mechanism to avoid upload of unencrypted data.

Steps to Reproduce

  • Setup an account A with AWS Organizations and the below SCP
  • Add another account B to this organization
  • Make sure to attach the below policy so that it applies to Account B.
  • agc account activate -> will fail due to S3 encryption header enforced
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": [
            "AES256",
            "aws:kms"
          ]
        },
        "ArnNotLike": {
          "aws:PrincipalARN": [
            "arn:aws:iam::*:role/XXX_*"
          ]
        }
      },
      "Action": [
        "s3:PutObject"
      ],
      "Resource": "*",
      "Effect": "Deny",
      "Sid": "DenyUnencryptedObjectUploads"
    },
    {
      "Condition": {
        "Null": {
          "s3:x-amz-server-side-encryption": "true"
        },
        "ArnNotLike": {
          "aws:PrincipalARN": [
            "arn:aws:iam::*:role/abc_logs",
            "arn:aws:iam::*:role/XXXX_*"
          ]
        }
      },
      "Action": "s3:PutObject",
      "Resource": "*",
      "Effect": "Deny",
      "Sid": "DenyIncorrectEncryptionHeader"
    }
  ]
}

Relevant Logs
11
//: # (The logs that where obtained by running the commands here. Please run the command with "-v" so that we can see the verbose logs.)

Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)

Screenshot 2022-11-23 at 09 28 30

Admin:~/environment $ agc account activate --vpc vpc-051647cf231f041b8 --subnets subnet-06f8288cdb3201075 --subnets subnet-0d060254c7bd29f95
2022-11-23T07:56:42Z 𝒊  Activating AGC with bucket '' and VPC 'vpc-051647cf231f041b8'
Bootstrapping CDK... [-o--] 27s                                                                                                                                                                                                                                                   
Activating account... [-o--] 2m34s                                                                                                                                                                                                                                                2022-11-23T07:59:43Z ✘  [WARNING] aws-cdk-lib.aws_ssm.StringParameterProps#type is deprecated.
2022-11-23T07:59:43Z ✘    - type will always be 'String'
2022-11-23T07:59:43Z ✘    This API will be removed in the next major release.
2022-11-23T07:59:43Z ✘  [WARNING] aws-cdk-lib.aws_ssm.ParameterType is deprecated.
2022-11-23T07:59:43Z ✘    these types are no longer used
2022-11-23T07:59:43Z ✘    This API will be removed in the next major release.
2022-11-23T07:59:43Z ✘  [WARNING] aws-cdk-lib.aws_ssm.ParameterType#STRING is deprecated.
2022-11-23T07:59:43Z ✘    
2022-11-23T07:59:43Z ✘    This API will be removed in the next major release.
2022-11-23T07:59:43Z ✘  [Warning at /Agc-Core/InfraSubnet0] No routeTableId was provided to the subnet 'subnet-06f8288cdb3201075'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
2022-11-23T07:59:43Z ✘  [Warning at /Agc-Core/InfraSubnet1] No routeTableId was provided to the subnet 'subnet-0d060254c7bd29f95'. Attempting to read its .routeTable.routeTableId will return null/undefined. (More info: https://github.com/aws/aws-cdk/pull/3171)
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘  ✨  Synthesis time: 10.4s
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘  Agc-Core: building assets...
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘  current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-deploy-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘  [0%] start: Building c409e6c5845f1f349df8cd84e160bf6f1c35d2b060b63e1f032f9bd39d4542cc:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [0%] start: Building 6ddcf10002539818a9256eff3fb2b22aa09298d8f946e26ba121c175a600c44e:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [0%] start: Building 42db86a487252e250546426e8c997e1fb797909d9e01db53902832b49909ced7:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [0%] start: Building 11e46d2fb8496407a00a5c8346ce8eb081821be164ecad1e9978d6646fad053a:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [0%] start: Building 8a2563dbc0ba4f7145d44accf5bbae6d797dd375f00bfa4221f516097125c28d:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [20%] success: Built c409e6c5845f1f349df8cd84e160bf6f1c35d2b060b63e1f032f9bd39d4542cc:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [40%] success: Built 6ddcf10002539818a9256eff3fb2b22aa09298d8f946e26ba121c175a600c44e:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [60%] success: Built 42db86a487252e250546426e8c997e1fb797909d9e01db53902832b49909ced7:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [80%] success: Built 11e46d2fb8496407a00a5c8346ce8eb081821be164ecad1e9978d6646fad053a:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [100%] success: Built 8a2563dbc0ba4f7145d44accf5bbae6d797dd375f00bfa4221f516097125c28d:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘  Agc-Core: assets built
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘  Agc-Core: deploying...
2022-11-23T07:59:43Z ✘  current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-deploy-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘  [0%] start: Publishing c409e6c5845f1f349df8cd84e160bf6f1c35d2b060b63e1f032f9bd39d4542cc:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [0%] start: Publishing 6ddcf10002539818a9256eff3fb2b22aa09298d8f946e26ba121c175a600c44e:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [0%] start: Publishing 42db86a487252e250546426e8c997e1fb797909d9e01db53902832b49909ced7:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [0%] start: Publishing 11e46d2fb8496407a00a5c8346ce8eb081821be164ecad1e9978d6646fad053a:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [0%] start: Publishing 8a2563dbc0ba4f7145d44accf5bbae6d797dd375f00bfa4221f516097125c28d:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘  current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘  current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘  current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘  current credentials could not be used to assume 'arn:aws:iam::170156817504:role/cdk-agc-file-publishing-role-170156817504-us-east-1', but are for the right account. Proceeding anyway.
2022-11-23T07:59:43Z ✘  [20%] success: Published 42db86a487252e250546426e8c997e1fb797909d9e01db53902832b49909ced7:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [40%] success: Published 6ddcf10002539818a9256eff3fb2b22aa09298d8f946e26ba121c175a600c44e:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [60%] success: Published c409e6c5845f1f349df8cd84e160bf6f1c35d2b060b63e1f032f9bd39d4542cc:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [80%] success: Published 11e46d2fb8496407a00a5c8346ce8eb081821be164ecad1e9978d6646fad053a:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  [100%] success: Published 8a2563dbc0ba4f7145d44accf5bbae6d797dd375f00bfa4221f516097125c28d:170156817504-us-east-1
2022-11-23T07:59:43Z ✘  Agc-Core: creating CloudFormation changeset...
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:23 AM | REVIEW_IN_PROGRESS   | AWS::CloudFormation::Stack  | Agc-Core User Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:28 AM | CREATE_IN_PROGRESS   | AWS::CloudFormation::Stack  | Agc-Core User Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:33 AM | CREATE_IN_PROGRESS   | AWS::DynamoDB::Table        | Table (TableCD117FA1) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:33 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | vpc (vpcA2121C38) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:33 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:33 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | bucket (bucket43879C71) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:34 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | ComputeEnvImage (ComputeEnvImage84B45428) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:34 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | NumInfraSubnets (NumInfraSubnets35FDF285) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:34 AM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:34 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata          | CDKMetadata/Default (CDKMetadata) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:34 AM | CREATE_IN_PROGRESS   | AWS::Lambda::LayerVersion   | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:34 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:34 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | InfraSubnets (InfraSubnets06E8F9B3) 
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:34 AM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:35 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:35 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | bucket (bucket43879C71) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:35 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | vpc (vpcA2121C38) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:35 AM | CREATE_IN_PROGRESS   | AWS::CDK::Metadata          | CDKMetadata/Default (CDKMetadata) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:36 AM | CREATE_IN_PROGRESS   | AWS::DynamoDB::Table        | Table (TableCD117FA1) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  0/15 | 7:57:36 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | ComputeEnvImage (ComputeEnvImage84B45428) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  1/15 | 7:57:36 AM | CREATE_COMPLETE      | AWS::SSM::Parameter         | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53) 
2022-11-23T07:59:43Z ✘  Agc-Core |  2/15 | 7:57:36 AM | CREATE_COMPLETE      | AWS::CDK::Metadata          | CDKMetadata/Default (CDKMetadata) 
2022-11-23T07:59:43Z ✘  Agc-Core |  2/15 | 7:57:36 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  2/15 | 7:57:36 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | NumInfraSubnets (NumInfraSubnets35FDF285) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  2/15 | 7:57:36 AM | CREATE_IN_PROGRESS   | AWS::SSM::Parameter         | InfraSubnets (InfraSubnets06E8F9B3) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  3/15 | 7:57:36 AM | CREATE_COMPLETE      | AWS::SSM::Parameter         | bucket (bucket43879C71) 
2022-11-23T07:59:43Z ✘  Agc-Core |  4/15 | 7:57:36 AM | CREATE_COMPLETE      | AWS::SSM::Parameter         | vpc (vpcA2121C38) 
2022-11-23T07:59:43Z ✘  Agc-Core |  5/15 | 7:57:37 AM | CREATE_COMPLETE      | AWS::SSM::Parameter         | ComputeEnvImage (ComputeEnvImage84B45428) 
2022-11-23T07:59:43Z ✘  Agc-Core |  6/15 | 7:57:37 AM | CREATE_COMPLETE      | AWS::SSM::Parameter         | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F) 
2022-11-23T07:59:43Z ✘  Agc-Core |  7/15 | 7:57:37 AM | CREATE_COMPLETE      | AWS::SSM::Parameter         | InfraSubnets (InfraSubnets06E8F9B3) 
2022-11-23T07:59:43Z ✘  Agc-Core |  8/15 | 7:57:37 AM | CREATE_COMPLETE      | AWS::SSM::Parameter         | NumInfraSubnets (NumInfraSubnets35FDF285) 
2022-11-23T07:59:43Z ✘  Agc-Core |  8/15 | 7:57:43 AM | CREATE_IN_PROGRESS   | AWS::Lambda::LayerVersion   | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core |  9/15 | 7:57:43 AM | CREATE_COMPLETE      | AWS::Lambda::LayerVersion   | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C) 
2022-11-23T07:59:43Z ✘  Agc-Core | 10/15 | 7:57:52 AM | CREATE_COMPLETE      | AWS::IAM::Role              | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) 
2022-11-23T07:59:43Z ✘  Agc-Core | 10/15 | 7:57:54 AM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) 
2022-11-23T07:59:43Z ✘  Agc-Core | 10/15 | 7:57:55 AM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core | 11/15 | 7:58:00 AM | CREATE_COMPLETE      | AWS::DynamoDB::Table        | Table (TableCD117FA1) 
2022-11-23T07:59:43Z ✘  Agc-Core | 12/15 | 7:58:13 AM | CREATE_COMPLETE      | AWS::IAM::Policy            | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) 
2022-11-23T07:59:43Z ✘  Agc-Core | 12/15 | 7:58:14 AM | CREATE_IN_PROGRESS   | AWS::Lambda::Function       | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) 
2022-11-23T07:59:43Z ✘  Agc-Core | 12/15 | 7:58:20 AM | CREATE_IN_PROGRESS   | AWS::Lambda::Function       | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:58:26 AM | CREATE_COMPLETE      | AWS::Lambda::Function       | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:58:28 AM | CREATE_IN_PROGRESS   | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) 
2022-11-23T07:59:43Z ✘  13/15 Currently in progress: Agc-Core, BatchArtifactsCustomResourceAA86556A
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:09 AM | CREATE_IN_PROGRESS   | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) Resource creation Initiated
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:09 AM | CREATE_FAILED        | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:10 AM | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack  | Agc-Core The following resource(s) failed to create: [BatchArtifactsCustomResourceAA86556A]. Rollback requested by user.
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::DynamoDB::Table        | Table (TableCD117FA1) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::SSM::Parameter         | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::SSM::Parameter         | bucket (bucket43879C71) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::SSM::Parameter         | NumInfraSubnets (NumInfraSubnets35FDF285) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::SSM::Parameter         | vpc (vpcA2121C38) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::SSM::Parameter         | ComputeEnvImage (ComputeEnvImage84B45428) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::SSM::Parameter         | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::SSM::Parameter         | InfraSubnets (InfraSubnets06E8F9B3) 
2022-11-23T07:59:43Z ✘  Agc-Core | 13/15 | 7:59:22 AM | DELETE_IN_PROGRESS   | AWS::CDK::Metadata          | CDKMetadata/Default (CDKMetadata) 
2022-11-23T07:59:43Z ✘  Agc-Core | 12/15 | 7:59:23 AM | DELETE_COMPLETE      | AWS::SSM::Parameter         | WesAdapterZipKeyParameter (WesAdapterZipKeyParameterCE036B53) 
2022-11-23T07:59:43Z ✘  Agc-Core | 11/15 | 7:59:23 AM | DELETE_COMPLETE      | AWS::SSM::Parameter         | bucket (bucket43879C71) 
2022-11-23T07:59:43Z ✘  Agc-Core | 10/15 | 7:59:23 AM | DELETE_COMPLETE      | AWS::SSM::Parameter         | vpc (vpcA2121C38) 
2022-11-23T07:59:43Z ✘  Agc-Core |  9/15 | 7:59:24 AM | DELETE_COMPLETE      | AWS::CDK::Metadata          | CDKMetadata/Default (CDKMetadata) 
2022-11-23T07:59:43Z ✘  Agc-Core |  8/15 | 7:59:24 AM | DELETE_COMPLETE      | AWS::SSM::Parameter         | NumInfraSubnets (NumInfraSubnets35FDF285) 
2022-11-23T07:59:43Z ✘  Agc-Core |  7/15 | 7:59:24 AM | DELETE_COMPLETE      | AWS::SSM::Parameter         | ComputeEnvImage (ComputeEnvImage84B45428) 
2022-11-23T07:59:43Z ✘  Agc-Core |  6/15 | 7:59:24 AM | DELETE_COMPLETE      | AWS::SSM::Parameter         | installed-artifacts--s3-root-url (installedartifactss3rooturl8C1CE61F) 
2022-11-23T07:59:43Z ✘  Agc-Core |  5/15 | 7:59:24 AM | DELETE_COMPLETE      | AWS::SSM::Parameter         | InfraSubnets (InfraSubnets06E8F9B3) 
2022-11-23T07:59:43Z ✘  Agc-Core |  6/15 | 7:59:24 AM | DELETE_COMPLETE      | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) 
2022-11-23T07:59:43Z ✘  Agc-Core |  6/15 | 7:59:26 AM | DELETE_IN_PROGRESS   | AWS::Lambda::Function       | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) 
2022-11-23T07:59:43Z ✘  Agc-Core |  5/15 | 7:59:33 AM | DELETE_COMPLETE      | AWS::Lambda::Function       | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C81C01536) 
2022-11-23T07:59:43Z ✘  Agc-Core |  5/15 | 7:59:33 AM | DELETE_IN_PROGRESS   | AWS::IAM::Policy            | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) 
2022-11-23T07:59:43Z ✘  Agc-Core |  5/15 | 7:59:33 AM | DELETE_IN_PROGRESS   | AWS::Lambda::LayerVersion   | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C) 
2022-11-23T07:59:43Z ✘  Agc-Core |  4/15 | 7:59:34 AM | DELETE_COMPLETE      | AWS::IAM::Policy            | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole/DefaultPolicy (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRoleDefaultPolicy88902FDF) 
2022-11-23T07:59:43Z ✘  Agc-Core |  3/15 | 7:59:34 AM | DELETE_COMPLETE      | AWS::DynamoDB::Table        | Table (TableCD117FA1) 
2022-11-23T07:59:43Z ✘  Agc-Core |  3/15 | 7:59:35 AM | DELETE_IN_PROGRESS   | AWS::IAM::Role              | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) 
2022-11-23T07:59:43Z ✘  Agc-Core |  2/15 | 7:59:35 AM | DELETE_COMPLETE      | AWS::Lambda::LayerVersion   | BatchArtifacts/AwsCliLayer (BatchArtifactsAwsCliLayer1CC86C5C) 
2022-11-23T07:59:43Z ✘  Agc-Core |  1/15 | 7:59:36 AM | DELETE_COMPLETE      | AWS::IAM::Role              | Custom::CDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756C/ServiceRole (CustomCDKBucketDeployment8693BB64968944B69AAFB0CC9EB8756CServiceRole89A01265) 
2022-11-23T07:59:43Z ✘  Agc-Core |  2/15 | 7:59:36 AM | ROLLBACK_COMPLETE    | AWS::CloudFormation::Stack  | Agc-Core 
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘  Failed resources:
2022-11-23T07:59:43Z ✘  Agc-Core | 7:59:09 AM | CREATE_FAILED        | Custom::CDKBucketDeployment | BatchArtifacts/CustomResource/Default (BatchArtifactsCustomResourceAA86556A) Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘   ❌  Agc-Core failed: Error: The stack named Agc-Core failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Command  '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘      at FullCloudFormationDeployment.monitorDeployment (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/api/deploy-stack.ts:496:13)
2022-11-23T07:59:43Z ✘      at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-11-23T07:59:43Z ✘      at deployStack2 (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:241:24)
2022-11-23T07:59:43Z ✘      at /home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/deploy.ts:39:11
2022-11-23T07:59:43Z ✘      at run (/home/ec2-user/.agc/cdk/node_modules/p-queue/dist/index.js:163:29)
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘   ❌ Deployment failed: Error: Stack Deployments Failed: Error: The stack named Agc-Core failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘      at deployStacks (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/deploy.ts:61:11)
2022-11-23T07:59:43Z ✘      at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-11-23T07:59:43Z ✘      at CdkToolkit.deploy (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:314:7)
2022-11-23T07:59:43Z ✘      at initCommandLine (/home/ec2-user/.agc/cdk/node_modules/aws-cdk/lib/cli.ts:357:12)
2022-11-23T07:59:43Z ✘  
2022-11-23T07:59:43Z ✘  Stack Deployments Failed: Error: The stack named Agc-Core failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Command '['/opt/awscli/aws', 's3', 'sync', '/tmp/tmp_9zlwqfu/contents', 's3://agc-170156817504-us-east-1/artifacts', '--metadata', '{"idempotency-key":"1.5.2"}', '--metadata-directive', 'REPLACE']' returned non-zero exit status 1. (RequestId: 3312c559-16ea-419a-b8f2-ec3378ca0657)
2022-11-23T07:59:43Z ✘   error="exit status 1"
Error: an error occurred invoking 'account activate'
with variables: {bucketName: vpcId:vpc-051647cf231f041b8 publicSubnets:false customTags:map[] subnets:[subnet-06f8288cdb3201075 subnet-0d060254c7bd29f95] amiId:}
caused by: exit status 1

Expected Behavior

The following modifcation of the AGC code core-stack.ts resolves this issue by sending along the encryption header when uploading:

   new BucketDeployment(this, "BatchArtifacts", {
      sources: [Source.asset(path.join(__dirname, "../artifacts"))],
      destinationBucket: this.bucket,
      destinationKeyPrefix: "artifacts",
      prune: false,
      metadata: {
        "idempotency-key": props.idempotencyKey,
      },
      serverSideEncryption: ServerSideEncryption.AES_256,
    });

Actual Behavior

Screenshots

Additional Context

Suggested Implementation:
see the TODOs in the following branch main...ArlindNocaj:amazon-genomics-cli:feature/sse-headers

Operating System:
AGC Version:
Was AGC setup with a custom bucket:
Was AGC setup with a custom VPC:
@ArlindNocaj ArlindNocaj added the bug Something isn't working label Nov 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ArlindNocaj