Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a short-lived cache for all requests in case of network problems in connection to AWS #7

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 98 additions & 0 deletions .github/workflows/docker-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
name: Docker

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

on:
schedule:
- cron: '41 23 * * *'
push:
branches: [ "master" ]
# Publish semver tags as releases.
tags: [ 'v*.*.*' ]
pull_request:
branches: [ "master" ]

env:
# Use docker.io for Docker Hub if empty
REGISTRY: ghcr.io
# github.repository as <account>/<repo>
IMAGE_NAME: ${{ github.repository }}


jobs:
build:

runs-on: ubuntu-latest
permissions:
contents: read
packages: write
# This is used to complete the identity challenge
# with sigstore/fulcio when running outside of PRs.
id-token: write

steps:
- name: Checkout repository
uses: actions/checkout@v4

# Install the cosign tool except on PR
# https://github.com/sigstore/cosign-installer
- name: Install cosign
if: github.event_name != 'pull_request'
uses: sigstore/cosign-installer@59acb6260d9c0ba8f4a2f9d9b48431a222b68e20 #v3.5.0
with:
cosign-release: 'v2.2.4'

# Set up BuildKit Docker container builder to be able to build
# multi-platform images and export cache
# https://github.com/docker/setup-buildx-action
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f95db51fddba0c2d1ec667646a06c2ce06100226 # v3.0.0

# Login against a Docker registry except on PR
# https://github.com/docker/login-action
- name: Log into registry ${{ env.REGISTRY }}
if: github.event_name != 'pull_request'
uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3.0.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

# Extract metadata (tags, labels) for Docker
# https://github.com/docker/metadata-action
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@96383f45573cb7f253c731d3b3ab81c87ef81934 # v5.0.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image
id: build-and-push
uses: docker/build-push-action@0565240e2d4ab88bba5387d719585280857ece09 # v5.0.0
with:
context: .
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max

# Sign the resulting Docker image digest except on PRs.
# This will only write to the public Rekor transparency log when the Docker
# repository is public to avoid leaking data. If you would like to publish
# transparency data even for private images, pass --force to cosign below.
# https://github.com/sigstore/cosign
- name: Sign the published Docker image
if: ${{ github.event_name != 'pull_request' }}
env:
# https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-an-intermediate-environment-variable
TAGS: ${{ steps.meta.outputs.tags }}
DIGEST: ${{ steps.build-and-push.outputs.digest }}
# This step uses the identity token to provision an ephemeral certificate
# against the sigstore community Fulcio instance.
run: echo "${TAGS}" | xargs -I {} cosign sign --yes {}@${DIGEST}
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [2.0.2] - 2024-09-30
### Changed
- Explicit env source
- Crontab conf path changed

## [2.0.0] - 2024-08-24

### Added
- Support for AWS WebIdentity Token
- Support for AWS EC2 metadata
- Log of the AWS identity used
- Support to set custom renewal interval
- Fallback to AWS_REGION, AWS_DEFAULT_REGION and region in ECR URL

### Changed
- Upgraded to OpenResty 1.21.4.1
- Upgraded AWS CLI to 1.34.21
- Environment variables names simplified
- Cleaned up the repository structure
25 changes: 11 additions & 14 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,25 +1,22 @@
FROM openresty/openresty:1.19.9.1-12-alpine
FROM openresty/openresty:1.21.4.1-0-alpine

USER root

RUN apk add -v --no-cache bind-tools python3 py-pip py3-urllib3 py3-colorama supervisor \
&& mkdir /cache \
&& addgroup -g 110 nginx \
&& adduser -u 110 -D -S -h /cache -s /sbin/nologin -G nginx nginx \
&& pip install --upgrade pip awscli==1.11.183 \
&& apk -v --purge del py-pip
RUN apk add -v --no-cache bind-tools python3 py-pip py3-urllib3 py3-colorama supervisor
RUN mkdir /cache
RUN mkdir /etc/crontab
RUN addgroup -g 110 nginx && adduser -u 110 -D -S -h /cache -s /sbin/nologin -G nginx nginx
RUN pip install --upgrade pip awscli==1.34.21

COPY files/startup.sh files/renew_token.sh files/health-check.sh /
COPY files/ecr.ini /etc/supervisor.d/ecr.ini
COPY files/root /etc/crontabs/root
COPY scripts/startup.sh /
COPY scripts/renew_token.sh /
COPY config/supervisord/programs.ini /etc/supervisor.d/programs.ini

COPY files/nginx.conf /usr/local/openresty/nginx/conf/nginx.conf
COPY files/ssl.conf /usr/local/openresty/nginx/conf/ssl.conf
COPY config/nginx/nginx.conf /usr/local/openresty/nginx/conf/nginx.conf
COPY config/nginx/ssl.conf /usr/local/openresty/nginx/conf/ssl.conf

ENV PORT 5000
RUN chmod a+x /startup.sh /renew_token.sh

HEALTHCHECK --interval=5s --timeout=5s --retries=3 CMD /health-check.sh

ENTRYPOINT ["/startup.sh"]
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisord.conf"]
21 changes: 0 additions & 21 deletions LICENSE

This file was deleted.

87 changes: 40 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,49 @@
<p align="left">
<a href="https://hub.docker.com/r/esailors/aws-ecr-http-proxy" alt="Pulls">
<img src="https://img.shields.io/docker/pulls/esailors/aws-ecr-http-proxy" /></a>
<a href="https://www.esailors.de" alt="Maintained">
<img src="https://img.shields.io/maintenance/yes/2022.svg" /></a>

</p>

# aws-ecr-http-proxy

A very simple nginx push/pull proxy that forwards requests to AWS ECR and caches the responses locally.

### Differences Between Fork and Upstream Repository

- Added support for AWS WebIdentity Token
- Added support for AWS EC2 metadata
- Added log of the AWS identity used
- Added support to set custom renewal interval
- Fallback to AWS_REGION, AWS_DEFAULT_REGION and region in ECR URL
- Upgraded to OpenResty 1.21.4.1
- Upgraded AWS CLI to 1.34.21
- Environment variables names simplified
- Cleaned up the repository structure

### Configuration:
The proxy is packaged in a docker container and can be configured with following environment variables:

| Environment Variable | Description | Status | Default |
| :---------------------------------: | :--------------------------------------------: | :-------------------------------: | :--------: |
| `AWS_REGION` | AWS Region for AWS ECR | Required | |
| `AWS_ACCESS_KEY_ID` | AWS Account Access Key ID | Optional | |
| `AWS_SECRET_ACCESS_KEY` | AWS Account Secret Access Key | Optional | |
| `AWS_USE_EC2_ROLE_FOR_AUTH` | Set this to true if we do want to use aws roles for authentication instead of providing the secret and access keys explicitly | Optional | |
| `UPSTREAM` | URL for AWS ECR | Required | |
| `ECR` | URL for AWS ECR | Required | |
| `RESOLVER` | DNS server to be used by proxy | Required | |
| `PORT` | Port on which proxy listens | Required | |
| `CACHE_MAX_SIZE` | Maximum size for cache volume | Optional | `75g` |
| `CACHE_KEY` | Cache key used for the content by nginx | Optional | `$uri` |
| `ENABLE_SSL` | Used to enable SSL/TLS for proxy | Optional | `false` |
| `REGISTRY_HTTP_TLS_KEY` | Path to TLS key in the container | Required with TLS | |
| `REGISTRY_HTTP_TLS_CERTIFICATE` | Path to TLS cert in the container | Required with TLS | |
| `CACHE_MAX_SIZE` | Maximum size for cache volume | Optional | `75g` |
| `CACHE_KEY` | Cache key used for the content by nginx | Optional | `$uri` |
| `RENEW_INTERVAL_HOURS` | Interval for renewing the AWS credentials | Optional | `6` |
| `ENABLE_SSL` | Used to enable SSL/TLS for proxy | Optional | `false` |
| `SSL_KEY` | Path to TLS key in the container | Required with SSL | |
| `SSL_CERTIFICATE` | Path to TLS cert in the container | Required with SSL | |


AWS identity can be passed:
- using environment variables
- using AWS credentials file (mounted in the container)
- using WebIdentity Token (mounted in the container)
- on AWS EC2 via metadata

If `AWS_REGION` is not set, it will be deduced from ECR URL.

| Environment Variable | Description | Status | Default |
| :---------------------------------: | :--------------------------------------------: | :-------------------------------: | :--------: |
| `AWS_REGION` | Region | Optional | |
| `AWS_ACCESS_KEY_ID` | Access key | Optional | |
| `AWS_SECRET_ACCESS_KEY` | Secret key | Optional | |


### Example:

Expand All @@ -37,41 +54,17 @@ docker run -d --name docker-registry-proxy --net=host \
-v /registry/key.pem:/opt/ssl/key.pem \
-e PORT=5000 \
-e RESOLVER=8.8.8.8 \
-e UPSTREAM=https://XXXXXXXXXX.dkr.ecr.eu-central-1.amazonaws.com \
-e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
-e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
-e AWS_REGION=${AWS_DEFAULT_REGION} \
-e ECR=https://XXXXXXXXXX.dkr.ecr.eu-central-1.amazonaws.com \
-e CACHE_MAX_SIZE=100g \
-e ENABLE_SSL=true \
-e REGISTRY_HTTP_TLS_KEY=/opt/ssl/key.pem \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/opt/ssl/certificate.pem \
esailors/aws-ecr-http-proxy:latest
-e SSL_KEY=/opt/ssl/key.pem \
-e SSL_CERTIFICATE=/opt/ssl/certificate.pem \
ghcr.io/dreamlab/aws-ecr-http-proxy:master
```

If you ran this command on "registry-proxy.example.com" you can now get your images using `docker pull registry-proxy.example.com:5000/repo/image`.

### Deploying the proxy

#### Deploying with ansible

Modify the ansible role [variables](https://github.com/eSailors/aws-ecr-http-proxy/tree/master/roles/docker-registry-proxy/defaults) according to your need and run the playbook as follow:
```sh
ansible-playbook -i hosts playbook-docker-registry-proxy.yaml
```
In case you want to enable SSL/TLS please replace the SSL certificates with the valid ones in [roles/docker-registry-proxy/files/*.pem](https://github.com/eSailors/aws-ecr-http-proxy/tree/master/roles/docker-registry-proxy/files)

#### Deploying on Kubernetes with Helm
You can install on Kubernetes using the [community-maintained chart](https://github.com/evryfs/helm-charts/tree/master/charts/ecr-proxy) like this:

```shell
helm repo add evryfs-oss https://evryfs.github.io/helm-charts/
helm install evryfs-oss/ecr-proxy --name ecr-proxy --namespace ecr-proxy
```

See the [values-file](https://github.com/evryfs/helm-charts/blob/master/charts/ecr-proxy/values.yaml) for configuration parameters.


### Note on SSL/TLS
The proxy is using `HTTP` (plain text) as default protocol for now. So in order to avoid docker client complaining either:
- (**Recommended**) Enable SSL/TLS using `ENABLE_SSL` configuration. For that you will have to mount your **valid** certificate/key in the container and pass the paths using `REGISTRY_HTTP_TLS_*` variables.
- (**Recommended**) Enable SSL/TLS using `ENABLE_SSL` configuration. For that you will have to mount your **valid** certificate/key in the container and pass the paths using `SSL_*` variables.
- Mark the registry host as insecure in your client [deamon config](https://docs.docker.com/registry/insecure/).
32 changes: 14 additions & 18 deletions files/nginx.conf → config/nginx/nginx.conf
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
user nginx;
worker_processes 1;

events {
worker_connections 1024;
}

user nginx;
worker_processes 1;


http {
include mime.types;
default_type application/octet-stream;

keepalive_timeout 65;
sendfile on;

proxy_cache_path /cache/cache levels=1:2 keys_zone=cache:16m inactive=1y max_size=CACHE_MAX_SIZE use_temp_path=off;
resolver RESOLVER valid=30s;
proxy_cache_path /cache/cache levels=1:2 keys_zone=cache:16m inactive=1y max_size=__CACHE_MAX_SIZE__ use_temp_path=off;
resolver __RESOLVER__ valid=30s;

# this is necessary for us to be able to disable request buffering in all cases
proxy_http_version 1.1;
Expand All @@ -27,9 +28,9 @@ http {
}

server {
listen PORT SSL_LISTEN default_server;
listen __PORT__ __SSL_LISTEN__ default_server;

SSL_INCLUDE
__SSL_INCLUDE__

# Cache
add_header X-Cache-Status $upstream_cache_status;
Expand All @@ -54,15 +55,10 @@ http {
add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
add_header "Access-Control-Allow-Origin" "*";

# health check
location /healthz {
return 200;
}

location / {
set $url UPSTREAM;
set $url __ECR__;
proxy_pass $url;
proxy_redirect $url SCHEME://$host:PORT;
proxy_redirect $url __SCHEME__://$host:__PORT__;

# Add AWS ECR authentication headers
proxy_set_header X-Real-IP $remote_addr;
Expand All @@ -76,9 +72,9 @@ http {
# Content addressable files like blobs.
# https://docs.docker.com/registry/spec/api/#blob
location ~ ^/v2/.*/blobs/[a-z0-9]+:[a-f0-9]+$ {
set $url UPSTREAM;
set $url __ECR__;
proxy_pass $url;
proxy_redirect $url SCHEME://$host:PORT;
proxy_redirect $url __SCHEME__://$host:__PORT__;

# Add AWS ECR authentication headers
proxy_set_header X-Real-IP $remote_addr;
Expand All @@ -101,7 +97,7 @@ http {
set $saved_redirect_location '$upstream_http_location';
proxy_pass $saved_redirect_location;
proxy_cache cache;
proxy_cache_key CACHE_KEY;
proxy_cache_key __CACHE_KEY__;
proxy_cache_valid 200 1y;
proxy_cache_use_stale error timeout invalid_header updating
http_500 http_502 http_503 http_504;
Expand Down Expand Up @@ -140,7 +136,7 @@ http {
location /get_tags {
internal;
set_unescape_uri $req_uri $arg_req_uri;
proxy_pass UPSTREAM$req_uri;
proxy_pass __ECR__$req_uri;

# Add AWS ECR authentication headers
proxy_set_header X-Real-IP $remote_addr;
Expand Down
4 changes: 2 additions & 2 deletions files/ssl.conf → config/nginx/ssl.conf
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
ssl_certificate_key REGISTRY_HTTP_TLS_KEY;
ssl_certificate REGISTRY_HTTP_TLS_CERTIFICATE;
ssl_certificate_key __SSL_KEY__;
ssl_certificate __SSL_CERTIFICATE__;

ssl_protocols TLSv1.2;
ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
Expand Down
File renamed without changes.
Loading