Technical Architecture
This wiki will cover the high-level architecture for the Contributary website project, focusing on how the services are deployed and organized.
For information on the particular stacks used for developing the front and backend, see the Onboarding Guide.
The Contributary website will be hosted in Amazon Web Services (AWS), utilizing a number of its most popular and practical services to manage the application's overall operations.
- Route 53 - Managed Domain Name System (DNS) and hosted zone service, the main entry point into the production environment
- CloudFront - Content Delivery Network (CDN) service
- Simple Storage Service (S3) - Object (file) storage for hosting the frontend (leveraging S3's capabilities to act as a webhost) and data fetched from external services
- API Gateway - Service to manage "entry points" / endpoints to the backend APIs
- Elastic Cloud Compute (EC2) - Virtualized servers in the cloud that can scale as needed
- Lambda - "Functions" as a service, providing the capability to execute specific code only when needed, without having to provision a server
The development environment will comprise of two principal services, GitHub and CircleCI.
GitHub (the site you're on right now!) has many beneficial services in addition to code hosting that can be used for project management and facilitating collaboration and contributions. Issue Tracker, Wiki, Release Notes and code reviews are just a few of the key features the Providence Geeks website will use. More details will be available in each project's Contributing Guidelines. For an overview of some of the GitHub features that will be used, check out this deck!
Continuous Integration (CI) is the practice of continually building a project on each commit or merge into a specific branch (like master). Best practice is to additionally have all code reviews (pull requests) be built using a CI server, that will checkout the PR's branch and run the project's build against it.
CircleCI is a managed CI service that aims to make setting up CI easy. Projects will use a YAML file to version control the build configuration including the build environment (Operating System, build runtime, installing dependencies) and the build steps to execute (linting, packaging, unit testing, etc).
CircleCI will also be used to deploy code into S3 and EC2 for continuous delivery. For more information on this, see the Release Management documentation.
Referencing the above network diagram, this is the general flow of data starting from the top
- User loads the website URL https://www.contributary.community
- DNS will route the request to Route 53
- Route 53 will forward the request to CloudFront (prod distribution)
- CloudFront will route requests one of two ways: ** Requests for static assets will route to a bucket (www.pvdgeeks.org) in S3 hosting the static assets deployed from CircleCI ** Requests to api/* will be routed to API Gateway production staging.
- Hosting the frontend web application's built / bundled static assets (HTML, CSS, JS, etc), acting as the origin server for CloudFront
- Hosting data pulled out of external services (stored as
.jsonfiles)
- Routing specific API Requests to the appropriate micro service based on a domain responsibility, e.g. /api/events
API Gateway is setup with two stages, one as a stage environment, and one for the production environment. In this way, new endpoints can be tested in stage before they are made public in production.
- Serving as a reverse proxy to the GitHub API, available as /api/github/
It might make sense to front this by a memcache server
- Executing on a cron to fetch latest topology data from the repository dedicated repository in GitHub and put it in S3
- Acting as an endpoint (/api/topology) for API Gateway to return data put into S3
Staging will essentially be a second UI / frontend environment running on top of the same production infrastucture and services. Referencing the network diagram:
- User loads the website domain URL https://stage.contributary.community
- DNS will route the request to Route 53
- Route 53 will forward the request to CloudFront (stage distribution)
- CloudFront will route requests one of two ways: ** Requests for static assets will route to a bucket (stage.contributary.community) in S3 hosting the static assets deployed from CircleCI ** Requests to api/* will be routed to API Gateway stage staging.
The frontend webapp will be a simple static site intended to interact with backend APIs to get data to populate components with data to drive the user search expoerience. For more information of the actual features of the frontend webapp, please see the Application Design
- Language: JavaScript (ES2015+)
- Library / Framework: Create Evergreen App
- Build Tool: webpack
The backend API(s) will be responsible for managing access to data needed for client browsers and devices, using a RESTful URL scheme. The intent is for clients to interact with these APIs only, instead of having to access external data sources directly. For more information on the APIs available and how they work and how to use them, please see the API Design docs.
External services will be responsible for being the primary data sources for the services managed by API Gateway.
To start, the data topology (language / project selection) will live as a JSON object in GitHub at this repo. It will periodically get pulled into S3 so that it can be consumed by the frontend via API Gateway -> Lambda.
The GitHub API will be used to facilitate searching of repositories and issues in real time for users. The frontend won't interact with the API directly, it will be reverse proxied through a NodeJS server that will contain the needed credentials and business logic for commuincating with the GitHub API.