Engula Development Guide

[huachaohuang]

The design of Engula refers to a lot of background knowledge in multiple areas. To make it easier to involve in the development, let me collect the related materials here.

Practices

• The title of issues and pull requests should start with the module (and submodule) name. For example, docs, engula, engula/client, object-engine, object-engine/master, etc.

Inspiration

• API: inspired by Redis, and some programming languages like Rust, Python, etc.
• Architecture: inspired by Socrates, Aurora, Spanner, BigTable, TAO.
• Transaction: inspired by hybrid-logical-clocks (HLC)
• Stream engine: inspired by LogDevice, Delos
• Object engine: inspired by RocksDB, GFS, and DeltaLake.

Reference papers

• Socrates: The New SQL Server in the Cloud
• Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases
• Amazon Aurora: On Avoiding Distributed Consensus for I/Os, Commits, and Membership Changes
• Spanner: Google’s Globally-Distributed Database
• Bigtable: A Distributed Storage System for Structured Data
• The Google File System
• TAO: Facebook’s Distributed Data Store for the Social Graph
• Virtual Consensus in Delos
• Logical Physical Clocks and Consistent Snapshots in Globally Distributed Databases
• The Snowflake Elastic Data Warehouse
• Delta Lake: High-Performance ACID Table Storage Over Cloud

[tisonkun]

Thanks for sharing these materials! We can later push entries to dev guide.