Skip to content

Commit

Permalink
Update URL structure
Browse files Browse the repository at this point in the history
- Add support for parameters other then width and height
- 500x500 becomes widht:500|height:500
- Add convert parameter to solve the issue with double extensions in the
  original file. (the reason this change has been made).
- Upgrade Jest
- Update README.md
- Add CHANGELOG.md
  • Loading branch information
martijngastkemper committed Jul 12, 2023
1 parent 27bda35 commit c0fabeb
Show file tree
Hide file tree
Showing 38 changed files with 1,298 additions and 1,391 deletions.
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Upgrade to V4

The URL structure has changed to make it possible to add more options. The old URL structure is not supported anymore. But can easily be converted to the new structure.

Some examples:

- `/scaled/500x500/foobar.jpg` -> `/scaled/width:500_height:500/foobar.jpg`
- `/scaled/x400/foobar.jpg.webp` -> `/scaled/height:400_convert:webp/foobar.jpg.webp`

Notice that you have to add the `convert` parameter with a format argument and append the extension to the file name. Doing both prevents confusion when the original image has multiple extensions. And when downloading the image, the correct extension is used.

# Upgrade to V3

The function uses NodeJS 18 so update your NodeJS version in the deploy workflow.

Environment variable `AWS_REGION` has been removed. Use the Serverless flag `--region`. For example: `npx serverless deploy --region eu-central-1`.

# Upgrade to V2

Upgrading to this version requires a change in the S3 Static website configuration. "ReplaceKeyPrefixWith": "default/resize?key=" must become "ReplaceKeyPrefixWith": "default/resize?key=scaled". This compensates for the removed SCALED_FOLDER env var.
136 changes: 66 additions & 70 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,15 @@
- [Usage](#usage)
- [Resizing](#resizing)
- [File conversion](#file-conversion)
- [Installation and deployment](#installation-and-deployment)
- [Clone and install](#clone-and-install)
- [Configure](#configure)
- [Environment variables](#environment-variables)
- [Defining SERVERLESS_ROLE](#defining-serverless_role)
- [Deploy](#deploy)
- [Use the microservice as a redirect rule in the bucket](#use-the-microservice-as-a-redirect-rule-in-the-bucket)
- [Use a CloudFront failover origin group](#use-a-cloudfront-failover-origin-group)
- [Command-line usage](#command-line-usage)
- [Local image server](#local-image-server)
- [Versions](#versions)
- [Installation](#installation)
- [Local development server](#local-development-server)
- [Command-line usage locally](#command-line-usage-locally)
- [Testing](#testing)
- [Deploy Lambda function](#deploy-lambda-function)
- [Connect the Lambda to your bucket](#connect-the-lambda-function-to-your-bucket)
- [Use as CloudFront failover group](#use-as-cloudfront-failover-group)
- [Debugging](#debugging)
- [Contributions](#contributions)

## Usage
Expand All @@ -44,25 +42,27 @@ Subsequent requests to the bucket will therefore skip the Lambda function entire

The function will allow resizing of images, by width, height, or both dimensions. Example paths:

- `/scaled/500x500/foobar.jpg`: this would generate a 500x500 version of the image `foobar.jpg`.
- `/scaled/500x/foobar.jpg`: omitting the `height` would generate a 500px wide version of the image.
- `/scaled/x500/foobar.jpg`: omitting the `width` would generate a 500px high version of the image.
- `/scaled/width:500_height:500/foobar.jpg`: this would generate a 500x500 version of the image `foobar.jpg`.
- `/scaled/width:500/foobar.jpg`: omitting the `height` would generate a 500px wide version of the image.
- `/scaled/height:500/foobar.jpg`: omitting the `width` would generate a 500px high version of the image.

### File conversion

You can double up on the extension to force a different output format.
You can convert an image to a different format.

- `/scaled/500x500/foobar.jpg.webp`: this will create a **webp** version of the file `foobar.jpg`, scaled to 500x500.
- `/scaled/width:500_height:500_convert:webp/foobar.jpg.webp`: this will create a **webp** version of the file `foobar.jpg`, scaled to 500x500.

Notice that you have to add the `convert` parameter with a format argument and append the extension to the file name. Doing both prevents confusion when the original image has multiple extensions. And when downloading the image, the correct extension is used.

## Versions

This repository uses branches for every major version. `v1.0`, `v2.x`, `v3.x`, etc. New features should be added to the latest version branch. Bug fixes should be added to the oldest version branch that is affected.

When deploying use a git tag to pin the version. This prevents breaking changes from being deployed automatically.

## Installation and deployment
`CHANGELOG.md` contains a list of all versions and their changes.

### Clone and install
## Installation

Clone this repository, and install dependencies:

Expand All @@ -77,32 +77,59 @@ For future reference, the following one-liner is used to install Sharp:
npm_config_platform=linux npm_config_arch=x64 yarn add sharp
```

### Configure

Configure a `.env` file, based on `.env.example`.

```sh
cp .env.example .env
```

#### Environment variables

The following environment variables are mandatory:
Mandatory environment variables are:

- `BUCKET`: the bucket in which your images are stored.
- `DEPLOYMENT_BUCKET`: the bucket to hold your Serverless deploys. Required when deploying using Serverless.
- `PROJECT_NAME` : Give AWS resources a Project tag with this value, defaults to `SERVICE_NAME`.
- `SERVERLESS_ROLE`: the role assumed by the Lambda function.
- `SERVICE_NAME`: the name of the Lambda function, defaults to "imageScaler".
- `SERVICE_NAME`: the name of the Lambda function.

The following environment variables are optional:
Optional environment variables are:

- `IMAGE_ACL`: the ACL applied to generated images. Default is empty. Use `public-read` when this service is deployed as S3 bucket redirect rule. When it's a CloudFront origin, use the default value.
- `QUALITY`: the format option quality setting for all image types. (integer: 1-100)

##### Defining SERVERLESS_ROLE
## Local development server

This package includes a local image server, allowing you to test with this image scaler locally, completely offline and independent of an AWS setup.

```sh
yarn serve
```

Then use http://localhost:8888/scaled/width:500_height:500/foo/bar.jpg for your image requests.

Currently, you will get back a random image with the requested dimensions.

## Command-line usage locally

A small utility is provided to resize images from the command-line.
This is especially helpful if you want to quickly test Sharp output, or generate fixtures for the test suite.

```sh
node cli/resize-image.js my-source-image.jpg width:500_height:500_convert:webp my-output-image.jpg.webp
```

## Testing

You can run the unit tests using

```sh
yarn test
```

This tests a variety of actual image conversions against problems we encountered in the wild.

## Deploy Lambda function

Create a role which Lambda functions are allowed to assume. Add the following trust relationship:
Create an IAM role which Lambda functions are allowed to assume. Add the following trust relationship:

```json
{
Expand Down Expand Up @@ -141,17 +168,17 @@ The role must have the following policy attached:

This allows the Lambda function to read and write from the bucket.

### Deploy

Deploy using the Serverless framework:

```sh
npx serverless deploy --stage staging|production --region eu-central-1
npx serverless deploy --stage=staging|production --region eu-central-1
```

Note the URL in Serverless' terminal output.

The function can work with a Website Configuration Redirection Rule or as a CloudFront Origin. It's not necessary to configure both.
## Connect the Lambda function to your bucket

The function can work as a Website Configuration Redirection Rule or as a CloudFront Origin. Choose one of the two.

### Use the microservice as a redirect rule in the bucket

Expand Down Expand Up @@ -183,16 +210,16 @@ Note some things:
- The value for `Condition.KeyPrefixEquals` is whatever you've configured as `SCALED_FOLDER` in the environment variables.
- The value for `HttpErrorCodeReturnedEquals` might be `403` or `404` based on your bucket. 403 when the objects in the bucket aren't public, 404 when they are.

#### Let's see if it works!
Let's see if it works!

Upload an image to your bucket (make sure it's publicly readable), for example `foobar.jpg`.

Now access this URL and see if it's working: `https://<BUCKET_URL>/scaled/500x500/foobar.jpg.webp`.
Now access this URL and see if it's working: `https://<BUCKET_URL>/scaled/width:500_height:500/foobar.jpg.webp`.
If not, the first step in debugging is to go to the Lambda function in the AWS Console and check the CloudWatch logs.

Good luck!

### Use a CloudFront failover origin group
## Use as CloudFront failover origin group

In your AWS console, go to your CloudFront distribution. Under <strong>Origins</strong> to can add a second origin, beside your S3 bucket.

Expand All @@ -204,53 +231,22 @@ The last step is changing the "Origin and origin group" property of the behavior

Save and wait for AWS to deploy the changes.

#### Let's see if it works!
Let's see if it works!

Upload an image to your bucket, for example, `foobar.jpg`.

Now access this URL and see if it's working: `https://<CLOUDFRONT_DISTRIBUTION>/scaled/500x500/foobar.jpg.webp`. If not, the first step in debugging is to go to the Lambda function in the AWS Console and check the CloudWatch logs.

Good luck!

## Command-line usage

A small utility is provided to resize images from the command-line.
This is especially helpful if you want to quickly test Sharp output, or generate fixtures for the test suite.

### Usage

```sh
node cli/resize-image.js my-source-image.jpg 500x500 my-output-image.jpg
```

## Local image server

This package includes a local image server, allowing you to test with this image scaler locally, completely offline and independent of an AWS setup.

### Usage:

```sh
yarn serve
```

Then use http://localhost:8888/scaled/500x500/foo/bar.jpg for your image requests.
Now access this URL and see if it's working: `https://<CLOUDFRONT_DISTRIBUTION>/scaled/width:500_height:500/foobar.jpg.webp`. If not, the first step in debugging is to go to the Lambda function in the AWS Console and check the CloudWatch logs.

Currently you will get back a random image with the requested dimensions.
## Debugging

## Upgrade to V3
The Lambda function logs to CloudWatch. You can find the logs in the AWS Console.

The function uses NodeJS 18 so update your NodeJS version in the deploy workflow.

## Testing

You can run the unit tests using
If you can only use the API. Use the following command to get the logs:

```sh
yarn test
aws logs tail /aws/lambda/<SERVICE_NAME>-<STAGE>-main --follow
```

This tests a variety of actual image conversions against problems we encountered in the wild.

## Contributions

Contributions are very welcome! If you're changing any of the image conversion rules, make sure to add a unit test to document the intended behavior.
Expand Down
55 changes: 21 additions & 34 deletions cli/resize-image.js
Original file line number Diff line number Diff line change
@@ -1,46 +1,33 @@
const fs = require("fs");
const getInputFormat = require("../util/getInputFormat.js");
const getOutputFormat = require("../util/getOutputFormat.js");
const parseOptions = require("../util/parseOptions.js");
const resizeImage = require("../util/resizeImage.js");
const pathToParams = require("../util/pathToParams.js");
const toSharpOptions = require("../util/toSharpOptions.js");
const { basename } = require("path");

const fileInput = process.argv[2];
if (!fileInput) {
const sourcePath = process.argv[2];
if (!sourcePath) {
throw new Error("Please provide a path to an image.");
}
const desiredDimensions = process.argv[3];
if (!desiredDimensions) {
throw new Error("Please provide the dimensions to scale the image.");
}
const fileOutput = process.argv[4];
if (!fileOutput) {
throw new Error("Please provide a path to write the resized image.");

const optionsString = process.argv[3];
if (!optionsString) {
throw new Error(
"Please provide options to scale the image. For example: width:100"
);
}

if (!fs.existsSync(fileInput)) {
throw new Error(`${fileInput} does not exist.`);
const destinationPath = process.argv[4];
if (!destinationPath) {
throw new Error("Please provide a path to write the resized image.");
}

const [size, , outputFormat] = pathToParams(
`${desiredDimensions}/${basename(fileInput)}`,
["jpg", "gif", "png", "jfif", "webp"]
);
const input = fs.readFileSync(sourcePath);

// Note: both dimensions are optional, but either width or height should always be present.
const dimensions = size.split("x");
const width = dimensions[0] && parseInt(dimensions[0], 10);
const height = dimensions[1] && parseInt(dimensions[1], 10);
if (!width && !height) {
throw new Error(`Unable to deduce dimensions from "${size}".`);
}
const options = parseOptions(optionsString);

const options = toSharpOptions({
width,
height,
fit: "cover",
});
const inputFormat = getInputFormat(destinationPath, options.convert);
const outputFormat = getOutputFormat(destinationPath, options.convert);

const input = fs.readFileSync(fileInput);
resizeImage(input, outputFormat, options).then((buffer) =>
fs.writeFileSync(fileOutput, buffer)
);
resizeImage(input, inputFormat, outputFormat, options.width, options.height)
.then((buffer) => fs.writeFileSync(destinationPath, buffer))
.then(() => console.log(`Resized image written to ${destinationPath}`));
Loading

0 comments on commit c0fabeb

Please sign in to comment.