🐛 Create a aws.Config with region to be able to work different AWS partition (like gov cloud or china AWS partition)

[calvix]

towards giantswarm/roadmap#3254

if this makes sense to you I will make a PR upstream

test in china galaxy and on golem work as expected

[AndiDog]

Where was the bug here? You're now supplying the region explicitly, but I guess that should be provided by CAPA to every AWS "create {instance,NAT gateway,...}" request. What's the relation to the partition (which is aws or aws-cn, for instance)?

[calvix]

Where was the bug here? You're now supplying the region explicitly, but I guess that should be provided by CAPA to every AWS "create {instance,NAT gateway,...}" request. What's the relation to the partition (which is aws or aws-cn, for instance)?

The bug is not providing the region to the AWS client because if you do not do that on the client creation it will fail with INVALID credentials, as it cannot validate the credentials - since it is connecting to the AWS global and not connecting to the AWS CHINA partition

Access keys generated in the China partition are not valid access keys in AWS global and by trying to check the validity of keys without using the region(which will tell the AWS client which partition to use) it will always use AWS global for and in our case it will fail to validate them. Resulting in non-recoverable error which stops any reconcilation.

[calvix]

here is the error log, after lot of debugging and injecting a debug output I localized it into that function where the PR fixes it

 > controller="awscluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSCluster" AWSCluster="org-giantswarm/galaxy" namespace="org-giantswarm" name="galaxy" reconcileID="e4da8234-95e0-476b-868f-d7c4aeda498c"
E0306 10:02:50.109642       1 controller.go:324] "Reconciler error" err=<
	error getting infra provider cluster or control plane object: failed to create aws session: Failed to retrieve identity credentials: InvalidClientTokenId: The security token included in the request is invalid.
		status code: 403, request id: 391bd0ff-52b9-412d-8d03-82aa8d720bd5
 > controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="org-giantswarm/galaxy-control-plane-12c432c9-qrtwx" namespace="org-giantswarm" name="galaxy-control-plane-12c432c9-qrtwx" reconcileID="74834d11-ff72-464a-b63c-df993020ad38"
E0306 10:02:50.187222       1 controller.go:324] "Reconciler error" err=<
	error getting infra provider cluster or control plane object: failed to create aws session: Failed to retrieve identity credentials: InvalidClientTokenId: The security token included in the request is invalid.
		status code: 403, request id: 63ba0c98-c34e-42c4-a276-4fbccc855908
 > controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="org-giantswarm/galaxy-control-plane-12c432c9-hpgm5" namespace="org-giantswarm" name="galaxy-control-plane-12c432c9-hpgm5" reconcileID="9246fa92-2b5f-40ab-8e82-5ca5cc464133"
E0306 10:02:50.202148       1 controller.go:324] "Reconciler error" err=<
	error getting infra provider cluster or control plane object: failed to create aws session: Failed to retrieve identity credentials: InvalidClientTokenId: The security token included in the request is invalid.
		status code: 403, request id: 2ff4274b-1075-454f-ad2a-9002de92f501
 > controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="org-giantswarm/galaxy-control-plane-12c432c9-bdtfw" namespace="org-giantswarm" name="galaxy-control-plane-12c432c9-bdtfw" reconcileID="80073767-a7ff-44b0-b866-7ef1d16c676d"
E0306 10:08:18.496207       1 controller.go:324] "Reconciler error" err=<
	failed to create scope: failed to create aws session: Failed to retrieve identity credentials: InvalidClientTokenId: The security token included in the request is invalid.
		status code: 403, request id: b1c30992-fdce-48e9-a66b-f951a4fd223d
	sigs.k8s.io/cluster-api-provider-aws/v2/pkg/cloud/scope.NewClusterScope
		/workspace/pkg/cloud/scope/cluster.go:76
	sigs.k8s.io/cluster-api-provider-aws/v2/controllers.(*AWSClusterReconciler).Reconcile
		/workspace/controllers/awscluster_controller.go:173
	sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
		/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.1/pkg/internal/controller/controller.go:118
	sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
		/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.1/pkg/internal/controller/controller.go:314
	sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
		/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.1/pkg/internal/controller/controller.go:265
	sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
		/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.1/pkg/internal/controller/controller.go:226
	runtime.goexit
		/usr/local/go/src/runtime/asm_amd64.s:1650

[whites11]

this smells like a go mod tidy is needed?

[calvix]

I actually run go mod tidy and it did nothing

[calvix]

I will just remove this , maybe my IDE does something fishy

[whites11]

sounds legit

[AndiDog]

Sounds reasonable to me. I think this is hard to test. Let's include the hotfix in our fork, and then please try to upstream it. Maybe a maintainer has more experience with this part of the code and can tell if it's problematic.

Did you test this change with cluster creation in China and non-China? That should be the minimum we do before merging.

[calvix]

Sounds reasonable to me. I think this is hard to test. Let's include the hotfix in our fork, and then please try to upstream it. Maybe a maintainer has more experience with this part of the code and can tell if it's problematic.

Did you test this change with cluster creation in China and non-China? That should be the minimum we do before merging.

Yes, as stated in the first message I tested in China and on golem