[Question] Is there a way to access aws-vaults ecs-server from within docker containers?

[JimFawkes]

I am running a few services as docker containers. In order to test some functionality (e.g. accessing S3) I need to pass aws credentials to my containers via env vars generated by aws-vault. This requires a restart of the services every time the credentials expired.
Since I am too lazy to spend a few seconds every hour or so on restarting the docker containers, I naturally spent several hours trying to find a better way. Unfortunately I did not get very far, so I thought I'd ask here in case someone has solved this (or can point out how I am thinking about this the wrong way).

Question:

Is there a way to run aws-vault exec --ecs-server on the host and allow boto3 retrieve the creds from the metadata from within a container?

Note: I tried dockerizing aws-vault but encountered challenges with the macOS Keychain which I could not resolve so far.

[jstewmon]

I think you'll need to take an indirect route, since the AWS_CONTAINER_CREDENTIALS_FULL_URI env var has to point to 127.0.0.1 and there is no bridge network support with macos since docker is running in a VM.

But, docker resolves host.docker.internal within containers to the host address, so you can still access the ecs server, which we can leverage to define a credential_process inside the container:

# create a shell script to obtain ECS credentials and rewrite as credential_process (requires curl and jq)
cat <<EOF > "${HOME}/aws-credentials.sh" && chmod +x "${HOME}/aws-credentials.sh"
#!/usr/bin/env sh
curl -s \
-H"authorization: ${AWS_CONTAINER_AUTHORIZATION_TOKEN}" \
"${AWS_CONTAINER_CREDENTIALS_FULL_URI/127.0.0.1/host.docker.internal}" \
| jq '{Version: 1, AccessKeyId: .AccessKeyId, SecretAccessKey: .SecretAccessKey, SessionToken: .Token, Expiration: .Expiration}'
EOF

mkdir -p "${HOME}/.aws" && cat <<EOF > "${HOME}/.aws/credentials"
[default]
credential_process = "$HOME/aws-credentials.sh"
EOF

Of course, you can add those files to your container however you like - probably by coping them into the image of volume mounting.

I tested this in an alpine container by installing curl, jq and aws-cli (apk add -U aws-cli curl jq). Container started with:

aws-vault exec --ecs-server my-profile -- \
docker run -it \
-e AWS_CONTAINER_CREDENTIALS_FULL_URI \
-e AWS_CONTAINER_AUTHORIZATION_TOKEN \
alpine

[JimFawkes]

This is awesome, thanks a lot! I got it to work and learned a few things 🎉 .

Thanks again and have an excellent week!

[mtibben]

we can leverage to define a credential_process inside the container

Very clever.

Another route that might work: you could rewrite AWS_CONTAINER_CREDENTIALS_FULL_URI to use host.docker.internal then pass it through to the container

[dmerrick]

we can leverage to define a credential_process inside the container

Very clever.

Another route that might work: you could rewrite AWS_CONTAINER_CREDENTIALS_FULL_URI to use host.docker.internal then pass it through to the container

that won't work because you get the following error message: Can only retrieve metadata from these hosts: 169.254.170.2, localhost, 127.0.0.1

[samdoolin]

Another route that might work: you could rewrite AWS_CONTAINER_CREDENTIALS_FULL_URI to use host.docker.internal then pass it through to the container

that won't work because you get the following error message: Can only retrieve metadata from these hosts: 169.254.170.2, localhost, 127.0.0.1

There's an open issue to fix this aws/aws-sdk#562