Sample microservice to fetch, cache, and serve information about GitHub organizations and their public repositories.
The GitHub Org Microservice is intended to be a proof-of-concept (PoC) for building a small service that caches public repository listings for an organization on GitHub.
Fastify is used as the web framework for this PoC.
We wish to serve information about public repositories in a GitHub organization. Hopefully, we can do this at the lowest cost for ourselves and GitHub's API.
GitHub performs rate limiting on its public APIs. Luckily, ETags are provided to enable conditional requests and cache use. Thus, we will use a cache to keep a local copy of API results.
Google is the target organization for this PoC, and its scale presents several considerations...
- Repositories are updated often.
- Repositories are added/removed frequently.
- Repositories are numerous enough to require several pages from the paginated API.
Individual repositories are updated often, although their names are updated infrequently. Thus, we assume that cached repository information will remain relevant until an organization's ETag changes (at which point the org's repo cache becomes stale). This assumption will save us from making unnecessary requests to the API.
Two endpoints will be exposed by our microservice...
| Route | Description | Return Sample |
|---|---|---|
GET /v1/{org}/repositories |
Get repository info for an organization. | [{"name": "repo1"}, ..., {"name": "repoN"}] |
PUT /v1/{org}/repositories/to_file |
Save repository info in /tmp |
201 Created |
Assuming server is running on 127.0.0.1:8080 (i.e., the default if using npm start)...
# Get org public repos
curl 127.0.0.1:8080/v1/google/repositories
# Save org public repos to file on server
curl -v -X PUT 127.0.0.1:8080/v1/google/repositories/to_fileThe following diagram depicts an abstract representation of how data flows through the system when a client makes a request to the GET /v1/{org}/repositories endpoint.
The PUT /v1/{org}/repositories/to_file endpoint has a very similar data flow. Instead of returning the repository list to the client, the server simply saves the data locally and returns 201 Created. (The author will leave making a corresponding diagram as an exercise for the reader.)
This section describes how to set up a development environment for this project. Setup is also well documented in CI.
For Ease of use, a script is provided to install dependencies, build, && run the project in a single command. (Requires
sudo.)# Assumes `node14` and `npm` are installed ./install-build-run.sh
This section describes how to set up the project's build toolchain.
node14 was used in the development and testing of this project.
nvm is recommended for devs who manage multiple versions of NodeJS.
# Install nvm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash
source ~/.bashrc
# Install node14
nvm install 14
nvm use 14Transpilation of TypeScript into JavaScript requires tsc to be installed.
Assuming the author is using Ubuntu (or has the apt package manager)...
sudo apt install install -y node-typescriptTesting and linting this project requires eslint and jest to be installed globally.
npm i -g eslint jestUse npm to install project dependencies.
npm inpm run buildnpm run testnpm run lintContinuous integration is performed with GitHub Actions. Process is defined by the workflow manifest. Results from pipeline runs may be viewed on GitHub.
Continuous delivery is not yet implemented, although it is on the roadmap.
| Decision | Brief Description |
|---|---|
| Fasitfy used as server framework | Fastify provides us with a lightweight web framework. |
| GitHub Actions for CI/CD | GitHub actions will be used for all CI/CD related tasks. |