Skip to content

Commit

Permalink
Merge pull request #614 from synnaxlabs/361-rfc-cesium-race-conditions
Browse files Browse the repository at this point in the history
361 rfc cesium race conditions
  • Loading branch information
pjdotson authored Jul 5, 2024
2 parents f8ee4f6 + 06907a0 commit 09326c2
Show file tree
Hide file tree
Showing 13 changed files with 854 additions and 141 deletions.
4 changes: 2 additions & 2 deletions docs/tech/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Luckily, there are many great open-source key-value databases available. Synnax
go-embedded [pebble](https://github.com/cockroachdb/pebble) which is the backing store
for
[cockroachdb](https://www.cockroachlabs.com/). On startup, Synnax opens the database and
let's pebble handle the rest. We'll return to this topic in the
lets pebble handle the rest. We'll return to this topic in the
[distribution layer](#12---layer-2---distribution).

#### 1.2.2 - Cesium - Time-Series Engine
Expand All @@ -76,7 +76,7 @@ component arrays, and finds the optimal location to store them on disk. To retri
frames as quickly as possible, cesium builds several indexing structures to reduce the
amount of disk seeks required to locate a specific data set.

As it's interface, cesium exposes two essential types: `Writer` and `Iterator`. A
As its interface, cesium exposes two essential types: `Writer` and `Iterator`. A
`Writer` is a transactional object that accepts frames and writes them to disk, only
committing them when explicitly told to do so. If multiple frames are written that
have overlapping time ranges, the `Writer` will fail to commit. This ensures that data
Expand Down
11 changes: 5 additions & 6 deletions docs/tech/rfc/0002-220518-aspen-distributed-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Aspen's design consists of two gossip layers:
Delta aims to provide dynamic cluster membership. This is more difficult to
accomplish if each node is required to know about _all_ other nodes in the
cluster before being initialized. This is the approach taken
by [etcd](https://etcd.io/). By using a gossip based network, Delta can provide
by [etcd](https://etcd.io/). By using a gossip-based network, Delta can provide
a cluster membership system that is dynamic and resilient to failure.

This cluster membership and state gossip is considered Layer 1. Layer 1 is
Expand Down Expand Up @@ -224,8 +224,8 @@ second message assigning it an ID. If the peer rejects the request or doesn't
respond, it attempts to send the request to another peer. This cycle continues
until a peer acknowledges the request or a preset threshold is reached.

The peer that accepts the **pledge** join request is known as the **
responsible**. This node is responsible for safely initiating the **pledge**.
The peer that accepts the **pledge** join request is known as the **responsible**.
This node is responsible for safely initiating the **pledge**.

### 2 - Propose an ID

Expand All @@ -242,8 +242,7 @@ If any node rejects the id, the **responsible** node will reissue the proposal
with an incremented value. This process continues until an ID is accepted. If
the **responsible** node tries to contact an unresponsive peer, it will reselect
a quorum of peers and try again. Once an ID is selected, the **responsible**
node will send it to the **
pledge**.
node will send it to the **pledge**.

### 3 - Disseminate New Node

Expand Down Expand Up @@ -275,7 +274,7 @@ protocol that disseminates kv updates and tombstones is known as layer 2.
**Host** - The node that is responsible for serving the kv operation to the
caller
(i.e. the node where `Get` or `Set` is called). \
**Leaseholder** - The only node that can accept writes for a particular key. \
**Leaseholder** - The only node that can accept writes for a particular key.

### 2 - Interface

Expand Down
2 changes: 2 additions & 0 deletions docs/tech/rfc/0016-231001-frame-flight-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,10 @@ array that can be unpacked at the receiver, and thus recreate the frame.

**Frame** - Data structure for wrapping primitive arrays with identifying
metadata into aligned arrays that streamline the telemetry transfer process. </b>

**Telemetry** - Data samples received from sensors and sent to actuators; typically
stored on Synnax server. More details available [here](../../../pluto). </b>

**Series** - A strongly typed collection of telemetry samples over a time range. The
fundamental unit of data transfer in Synnax server. </b>

Expand Down
978 changes: 845 additions & 133 deletions docs/tech/rfc/0019-240310-cesium-race-conditions.md

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 09326c2

Please sign in to comment.