Skip to content

An enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.

License

Notifications You must be signed in to change notification settings

wednesday-solutions/node-mongo-express

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Node Mongo Express

An enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.


Expert teams of digital product strategists, developers, and designers.


We’re always looking for people who value their work, so come and join us. We are hiring!

Node Mongo Express CI

Node Mongo Express CD



Pre-requisites

- yarn
- docker

Features

- Mongo support
- Docker support
- Rate limited APIs
- RBAC middleware using Auth0
- Sharding mongoDB collection support
- Paginated APIs
- Autogenerated APIs from mongoose models
- Built in slack alerting mechanism
- Suport for redis cache
- Support for aggregate caching
- Support for batch jobs in multi-container environment
- Support for circuit breakers
- Autogenerated swagger documentation
- Load testing using k6
- Support for i18n

Running Load tests

  • Install k6
  • Execute the following command: k6 run __tests__/__load__/script.js

Build and run docker container locally

- docker-compose down
- docker-compose build
- docker-compose up

Shard setup

Run the following script

./setup-shards/scripts/setup/base.sh

Take a look at this to create shards and replica sets.

Seeders

Run the following command to begin seeding

./seeders/seed.sh

How to start

- cd `node-mongo-express`
- yarn
- ./setup-shards/scripts/setup/base.sh
- cp .env.example .env.local
- ./seeders/seed.sh
- yarn start
- open browser to `localhost:9000` (port default to 9000)

API Documentation

Once you've to the server started check out the api documentation at /api-docs

Navigating the code base

Philosophy

When using NoSQLs you are optimising for read performance. We're doing this by denormalising data. There are multiple copies of the same data. For example

  • Orders contains purchasedProducts which contains Products. Instead of referencing here we embed
  • SupplierProducts contains embedded objects for both Suppliers and Products
  • StoreProducts contains embedded objects for both Stores and Products

This makes our application write heavy. Every time there is a change to a product we need to make a change to

  • SupplierProducts
  • StoreProducts
  • Products

Orders is not impacted since a change in the product after purchase will not affect the order.

However the application is able to perform extremely fast reads. 2 reasons for better performance is

  • shards
  • document embedding

NoSQLs are also good for handling large volumes of data. This is supported due to its ability to have shards. In this application we create 4 shards and the data is distributed amongst these shards.

These are the shard keys that we use

  • _id
    • Order
  • name
    • Products
    • Suppliers
    • Stores
      We got really good distribution across shards(24-26%) per shard after seeding 4 million records. It's possible to get a hot shard due to this but we're yet to see that.
  • productId
    • SupplierProducts
    • StoreProducts
      productId is chosen as the shard key since we anticipate that the queries for fetching all suppliers/stores that sell a particular product will be much more than fetching all products of a supplier/store.

About

An enterprise Mongo-Express REST API built using nodejs showcasing - Testing Strategy, mongoDB sharding, models, a REST API Interface, support for Redis, aggregation queries, aggregation caching, circuit-breakers, slack integration, RBAC, rate limited APIs and multi-container queues and schedulers.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages