Skip to content

An application with physical shards of the database

Notifications You must be signed in to change notification settings

margato/go-mysql-sharding

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

This project is a demonstration of an application with physical shards of the database, using Go. Sharding helps to distribute data across multiple databases to improve performance and scalability. In this example, customer data is distributed across three shards.

Sharding Database Architecture

Running the Application

  1. Clone the repository:
git clone https://github.com/margato/go-mysql-sharding.git
cd go-sharding
  1. Navigate to the infra folder and start the Docker containers:
cd infra
docker-compose up -d
  1. Set up the environment variables:

Ensure that your MySQL instances are running and set the environment variables accordingly, in the app path:

cd ../app
export SHARD_0_DSN="root:@tcp(localhost:3306)/customers?charset=utf8mb4&parseTime=True&loc=Local"
export SHARD_1_DSN="root:@tcp(localhost:3307)/customers?charset=utf8mb4&parseTime=True&loc=Local"
export SHARD_2_DSN="root:@tcp(localhost:3308)/customers?charset=utf8mb4&parseTime=True&loc=Local"
  1. Run the application:
go build
./go-mysql-sharding

Testing the Application

Create a New Customer

To create a new customer, use the following curl command:

curl -i -X POST http://localhost:8080/customers \
  -H "Content-Type: application/json" \
  -d '{"name": "John Doe"}'

Get Customer by ID

To get a customer by their ID, use the following curl command, replacing with the actual customer ID:

curl -i http://localhost:8080/customers/<REPLACE ID>

Results

We can bulk insert 1,000 customers with the code below:

for i in {1..1000}
do
  curl -i -X POST http://localhost:8080/customers \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"John Doe $i\"}"
  sleep 0.1 
done

After running the script, we can see the following distribution:

Shard Customers
0 342
1 327
2 331

Obs.: This results vary in each execution, given hashes are random.

About

An application with physical shards of the database

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages