Skip to content

Commit

Permalink
Read AWS credentials from .env (#87)
Browse files Browse the repository at this point in the history
* Read aws credentials from .env

* Bump version
  • Loading branch information
erxclau authored Aug 25, 2023
1 parent d6b6516 commit ad75663
Show file tree
Hide file tree
Showing 11 changed files with 854 additions and 808 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ yarn-error.log
!example.config.js

dist/*
.env
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ A collection of helper scripts that are used across The Michigan Daily's project

Run `yarn add --dev michigandaily/sink` to get the latest version.

If you want to install a specifc version, add a version tag at the end of the library name (e.g., `michigandaily/sink#v2.8.0`).
If you want to install a specifc version, add a version tag at the end of the library name (e.g., `michigandaily/sink#v2.9.0`).

## Google Drive fetch

Expand Down Expand Up @@ -114,13 +114,13 @@ For security purposes, the service account and associated client email should be

## AWS S3 deployment with cache invalidation

Create a configuration file. The file should have a `deployment` property with an object value. The value should include the following properties: `region`, `bucket`, `key`, `build`, and `profile`. The value can optionally include a `distribution` property.
Create a configuration file. The file should have a `deployment` property with an object value. The value should include the following properties: `region`, `bucket`, `key`, `build`. The value can optionally include `distribution` and `profile` properties.

- The `region` property specifies where the S3 bucket is located.
- The `bucket` property will be used to determine which S3 bucket to deploy to.
- The `key` property will be used to determine which sub-directory in the `bucket` to deploy to.
- The `build` property will be used to determine which directory's content will be deployed to S3.
- The `profile` property will be used as the name of the AWS credentials profile specified in `~/.aws/credentials`.
- The `profile` property will be used as the name of the AWS credentials profile specified in `~/.aws/credentials`. If a `profile` property is not specified, `sink` will attempt to read credentials from the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables.
- The `distribution` property specifies the S3 bucket's associated CloudFront distribution. This will be used to invalidate files if needed. If `distribution` is not specified, `sink` will attempt to find a CloudFront distribution associated with the `bucket`. If you do not want to invalidate the bucket's distribution, either set `distribution` as an empty string, `null`, or `false`.

> To Daily staffers: ask a managing online editor for access to AWS credentials. They will retrieve it for you from 1Password via a private link. You will receive a CSV file that contains "Access key ID" and "Secret access key" columns. If you do not already have a file located at `~/.aws/credentials`, create that file. Then populate it with the following:
Expand Down
9 changes: 5 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "sink",
"version": "2.8.0",
"version": "2.9.0",
"description": "Helper scripts for The Michigan Daily",
"main": "src/sink.js",
"bin": {
Expand All @@ -19,15 +19,16 @@
"private": true,
"type": "module",
"dependencies": {
"@aws-sdk/client-cloudfront": "^3.352.0",
"@aws-sdk/client-s3": "^3.352.0",
"@aws-sdk/credential-providers": "^3.352.0",
"@aws-sdk/client-cloudfront": "^3.398.0",
"@aws-sdk/client-s3": "^3.398.0",
"@aws-sdk/credential-providers": "^3.398.0",
"@googleapis/drive": "^5.1.0",
"@googleapis/sheets": "^4.0.2",
"archieml": "^0.5.0",
"chalk": "^5.2.0",
"commander": "^10.0.1",
"d3-dsv": "^3.0.1",
"dotenv": "^16.3.1",
"find-up": "^6.3.0",
"google-auth-library": "^8.8.0",
"html-entities": "^2.3.6",
Expand Down
29 changes: 26 additions & 3 deletions src/sink-deploy.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { readdirSync, lstatSync, createReadStream } from "node:fs";
import { join, extname, dirname, normalize, posix } from "node:path";
import { createHash } from "node:crypto";
import { createInterface } from "node:readline";
import { exit } from "node:process";

import { program, Argument } from "commander";
import chalk from "chalk";
Expand All @@ -18,7 +19,7 @@ import {
CreateInvalidationCommand,
ListDistributionsCommand,
} from "@aws-sdk/client-cloudfront";
import { fromIni } from "@aws-sdk/credential-providers";
import { fromIni, fromEnv } from "@aws-sdk/credential-providers";
import { lookup } from "mime-types";

import { load_config, success, fatal_error } from "./_utils.js";
Expand Down Expand Up @@ -110,7 +111,29 @@ const main = async ([platform], opts) => {
console.log("skipping build step");
}

const credentials = fromIni({ profile });
let credentials;

if (!!profile) {
credentials = fromIni({ profile });
} else {
console.log(
"no AWS credentials profile was specified. falling back to environment variables."
);
await import("dotenv/config");

if (
!!process.env.AWS_ACCESS_KEY_ID &&
!!process.env.AWS_SECRET_ACCESS_KEY
) {
credentials = fromEnv();
} else {
console.error(
"no AWS credentials were specified in the environment variables. exiting."
);
exit(1);
}
}

const client = new S3Client({ region, credentials });
const list = new ListObjectsCommand({ Bucket: bucket, Prefix: key });
const response = await client.send(list);
Expand Down Expand Up @@ -338,7 +361,7 @@ const main = async ([platform], opts) => {

if (process.argv[1] === self) {
program
.version("2.8.0")
.version("2.9.0")
.addArgument(
new Argument("<platform>", "platform to deploy to").choices([
"aws",
Expand Down
2 changes: 1 addition & 1 deletion src/sink-fetch.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ const main = async (opts) => {
const self = fileURLToPath(import.meta.url);
if (process.argv[1] === self) {
program
.version("2.8.0")
.version("2.9.0")
.option("-c, --config <path>", "path to config file")
.parse();

Expand Down
2 changes: 1 addition & 1 deletion src/sink-gdoc.js
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ const main = async (opts) => {
const self = fileURLToPath(import.meta.url);
if (process.argv[1] === self) {
program
.version("2.8.0")
.version("2.9.0")
.option("-c, --config <path>", "path to config file")
.parse();

Expand Down
2 changes: 1 addition & 1 deletion src/sink-gsheet.js
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ async function main(opts) {
const self = fileURLToPath(import.meta.url);
if (process.argv[1] === self) {
program
.version("2.8.0")
.version("2.9.0")
.option("-c, --config <path>", "path to config file")
.parse();

Expand Down
2 changes: 1 addition & 1 deletion src/sink-json.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ const main = async (opts) => {
const self = fileURLToPath(import.meta.url);
if (process.argv[1] === self) {
program
.version("2.8.0")
.version("2.9.0")
.option("-c, --config <path>", "path to config file")
.parse();

Expand Down
2 changes: 1 addition & 1 deletion src/sink-text.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ const main = async (opts) => {
const self = fileURLToPath(import.meta.url);
if (process.argv[1] === self) {
program
.version("2.8.0")
.version("2.9.0")
.option("-c, --config <path>", "path to config file")
.parse();

Expand Down
2 changes: 1 addition & 1 deletion src/sink.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import { program } from "commander";

program
.version("2.8.0")
.version("2.9.0")
.name("sink")
.description("Utility scripts")
.command("gdoc", "fetch ArchieML Google Doc into JSON file")
Expand Down
Loading

0 comments on commit ad75663

Please sign in to comment.