Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate v0.3.0 to IAM Role #11

Merged
merged 2 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.21.1-alpine3.18
FROM golang:1.23-alpine3.20
vishu-25 marked this conversation as resolved.
Show resolved Hide resolved

RUN apk update; apk add make git curl bash jq aws-cli

Expand Down
91 changes: 28 additions & 63 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,97 +1,62 @@
# proto api docs action
# Proto api docs action

This action clones protobuf files from a (public/private) github repository, generate swagger v2.0 api docs using buf & grpc-gateway's [protoc-gen-openapiv2](https://github.com/grpc-ecosystem/grpc-gateway/tree/master/protoc-gen-openapiv2) plugin, and finally upload the generated documentation to aws S3.
This action clones protobuf files from a (public/private) GitHub repository, generates Swagger v2.0 API docs using buf & grpc-gateway's [protoc-gen-openapiv2](https://github.com/grpc-ecosystem/grpc-gateway/tree/master/protoc-gen-openapiv2) plugin, and finally uploads the generated documentation to AWS S3.

## Inputs

### GIT_TOKEN:
### GIT_TOKEN

**Required** git token to use for cloning the repository containing the protobuf files
**Required** Git token to use for cloning the repository containing the protobuf files.

### PROTO_REPOSITORY:
### PROTO_REPOSITORY

**Required** protobuf files github repository including the org name
**Required** Protobuf files GitHub repository, including the org name.

### PROTO_BRANCH:
### PROTO_BRANCH

**Required** branch to clone
**Required** Branch to clone.

### MODULE_LIST_FILE_PATH:
### MODULE_LIST_FILE_PATH

**Required** path to file containing list of subdirectories of protos to clone
**Required** Path to the file containing the list of subdirectories of protos to clone.

### AWS_S3_BUCKET:
### AWS_S3_BUCKET

**Required** s3 bucket name
**Required** S3 bucket name.

### AWS_ACCESS_KEY_ID:
### AWS_ROLE_ARN

**Required** aws access key id
**Required** IAM role ARN to assume for S3 access.

### AWS_SECRET_ACCESS_KEY:
### AWS_ROLE_SESSION_NAME

**Required** aws access key secret
**Optional** Session name for the assumed role. Defaults to `s3-sync-session`.

### AWS_REGION:
### AWS_REGION

**Required** aws region
**Required** AWS region.

### DEST_DIR:
### DEST_DIR

**Required** destination s3 directory
**Required** Destination S3 directory.

another directory will be created in the destination directory with the name of the repository running this action. The generated documentation will be uploaded to this directory with branch name as the name of the file.
Another directory will be created in the destination directory with the name of the repository running this action. The generated documentation will be uploaded to this directory with the branch name as the name of the file.

## Outputs
NA
None

## Example usage
```yaml
- uses: actions/checkout@v2
- uses: razorpay/proto-api-docs-action@v0.2.2
- uses: razorpay/proto-api-docs-action@v0.3.0-iam-role
with:
GIT_TOKEN: ${{ secrets.GIT_TOKEN }}
PROTO_REPOSITORY: krantideep95/proto
PROTO_REPOSITORY: myorg/proto
PROTO_BRANCH: ${{ github.ref }}
MODULE_LIST_FILE_PATH: scripts/proto_modules
AWS_S3_BUCKET: apidocs
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_S3_BUCKET: my-bucket
AWS_ROLE_ARN: arn:aws:iam::123456789012:role/my-s3-access-role
AWS_ROLE_SESSION_NAME: s3-sync-session
AWS_REGION: ap-south-1
DEST_DIR: _docs
```

## FAQs

### Will it work with twirp?

Short Answer: Yes, with a few quirks.

Long Answer: proto-gen-openapiv2 plugin is meant to be used with grpc-gateway protobuf files. It is not meant to be used with twirp. twirp officially doesn't support a plugin to generate swagger api docs. By enabling a few options in plugin, this action can be used to generate swagger api docs for twirp. You can use following proto definitions in one of the files to make this compatible with twirp:
```protobuf
import "protoc-gen-openapiv2/options/annotations.proto";

option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_swagger) = {
schemes: HTTPS;
host: "base url";
base_path: "/twirp/";
security: {
security_requirement :{
key: "BasicAuth";
value: {};
}
}
security_definitions: {
security: {
key: "BasicAuth";
value: {
type: TYPE_BASIC;
}
}
}
};
```

### Should this be used in a common protobuf repo or different backend service specific repos ?

Usually, 1 backend service exposes different sub-domains' APIs in 1 deployment. For such cases, it makes sense to use this action separately in that backend service's repo.

44 changes: 23 additions & 21 deletions action.yml
Original file line number Diff line number Diff line change
@@ -1,46 +1,48 @@
name: 'proto api docs'
description: 'Generate swagger api docs from protobuf & push them to a s3 bucket'
description: 'Generate swagger api docs from protobuf & push them to an S3 bucket'
branding:
icon: 'edit-3'
color: 'gray-dark'
inputs:
GIT_TOKEN:
description: 'git token to use for cloning the repository containing the protobuf files'
description: 'Git token to use for cloning the repository containing the protobuf files'
required: true
PROTO_REPOSITORY:
description: 'protobuf files github repository including the org name'
description: 'Protobuf files GitHub repository including the org name'
required: true
PROTO_BRANCH:
description: 'branch to clone'
description: 'Branch to clone'
required: true
MODULE_LIST_FILE_PATH:
description: 'path to file containing list of subdirectories of protos to clone'
description: 'Path to file containing list of subdirectories of protos to clone'
required: true
AWS_S3_BUCKET:
description: 's3 bucket name'
description: 'S3 bucket name'
required: true
AWS_ACCESS_KEY_ID:
description: 'aws access key id'
required: true
AWS_SECRET_ACCESS_KEY:
description: 'aws access key secret'
AWS_ROLE_ARN:
description: 'IAM role ARN to assume for S3 access'
required: true
AWS_ROLE_SESSION_NAME:
description: 'Session name for the assumed role'
required: false
default: 's3-sync-session'
AWS_REGION:
description: 'aws region'
description: 'AWS region'
required: true
DEST_DIR:
description: 'destination s3 directory'
description: 'Destination S3 directory'
required: true
runs:
using: "docker"
image: 'Dockerfile'
args:
- ${{ inputs.GIT_TOKEN }}
${{ inputs.PROTO_REPOSITORY }}
${{ inputs.PROTO_BRANCH }}
${{ inputs.MODULE_LIST_FILE_PATH }}
${{ inputs.AWS_S3_BUCKET}}
${{ inputs.AWS_ACCESS_KEY_ID}}
${{ inputs.AWS_SECRET_ACCESS_KEY}}
${{ inputs.AWS_REGION}}
${{ inputs.DEST_DIR}}
- ${{ inputs.PROTO_REPOSITORY }}
- ${{ inputs.PROTO_BRANCH }}
- ${{ inputs.MODULE_LIST_FILE_PATH }}
- ${{ inputs.AWS_S3_BUCKET }}
- ${{ inputs.AWS_ROLE_ARN }}
- ${{ inputs.AWS_ROLE_SESSION_NAME }}
- ${{ inputs.AWS_REGION }}
- ${{ inputs.DEST_DIR }}

49 changes: 34 additions & 15 deletions upload_to_s3.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,51 +7,70 @@ if [ -z "$INPUT_AWS_S3_BUCKET" ]; then
exit 1
fi

if [ -z "$INPUT_AWS_ACCESS_KEY_ID" ]; then
echo "AWS_ACCESS_KEY_ID is not set. Quitting."
if [ -z "$INPUT_AWS_ROLE_ARN" ]; then
echo "AWS_ROLE_ARN is not set. Quitting."
exit 1
fi

if [ -z "$INPUT_AWS_SECRET_ACCESS_KEY" ]; then
echo "AWS_SECRET_ACCESS_KEY is not set. Quitting."
exit 1
if [ -z "$INPUT_AWS_ROLE_SESSION_NAME" ]; then
INPUT_AWS_ROLE_SESSION_NAME="s3-sync-session"
fi

# Default to us-east-1 if AWS_REGION not set.
if [ -z "$INPUT_AWS_REGION" ]; then
AWS_REGION="us-east-1"
else
AWS_REGION="$INPUT_AWS_REGION"
fi

# Override default AWS endpoint if user sets AWS_S3_ENDPOINT.
if [ -n "$INPUT_AWS_S3_ENDPOINT" ]; then
ENDPOINT_APPEND="--endpoint-url $INPUT_AWS_S3_ENDPOINT"
fi

# Create a dedicated profile for this action to avoid conflicts
# with past/future actions.
# https://github.com/jakejarvis/s3-sync-action/issues/1
# Assume the specified IAM role to get temporary credentials
CREDENTIALS=$(aws sts assume-role --role-arn "$INPUT_AWS_ROLE_ARN" \
--role-session-name "$INPUT_AWS_ROLE_SESSION_NAME" \
--region "$AWS_REGION" \
--query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' \
--output text)

if [ -z "$CREDENTIALS" ]; then
echo "Failed to assume role. Quitting."
exit 1
fi

# Parse the credentials
ACCESS_KEY_ID=$(echo "$CREDENTIALS" | awk '{print $1}')
SECRET_ACCESS_KEY=$(echo "$CREDENTIALS" | awk '{print $2}')
SESSION_TOKEN=$(echo "$CREDENTIALS" | awk '{print $3}')

# Configure AWS CLI with temporary credentials
aws configure --profile s3-sync-action <<-EOF > /dev/null 2>&1
${INPUT_AWS_ACCESS_KEY_ID}
${INPUT_AWS_SECRET_ACCESS_KEY}
${INPUT_AWS_REGION}
$ACCESS_KEY_ID
$SECRET_ACCESS_KEY
$AWS_REGION
text
EOF

# Set session token in the environment
export AWS_SESSION_TOKEN="$SESSION_TOKEN"

SOURCE_DIR=/_docs
# Sync using our dedicated profile and suppress verbose messages.
# All other flags are optional via the `args:` directive.
sh -c "aws s3 sync ${SOURCE_DIR} s3://${INPUT_AWS_S3_BUCKET}/${INPUT_DEST_DIR} \
--profile s3-sync-action \
--no-progress \
${INPUT_ENDPOINT_APPEND} $*"
${ENDPOINT_APPEND} $*"

# Clear out credentials after we're done.
# We need to re-run `aws configure` with bogus input instead of
# deleting ~/.aws in case there are other credentials living there.
# https://forums.aws.amazon.com/thread.jspa?threadID=148833
aws configure --profile s3-sync-action <<-EOF > /dev/null 2>&1
null
null
null
text
EOF

# Unset session token
unset AWS_SESSION_TOKEN