sea-tale is inspired by secure-scuttlebutt, and intends to extend the concept for use in apps that are not social networks.
sea-tale is based around the concept of personal blockchains. A personal blockchain (or 'chain') is a chain of messages published by a single node. Each message is signed with the node's private key, and contains the hash of the next message. This allows anyone with a node's public key to verify that the messages came from that node, and that they are an unbroken sequence. Nodes syncronize chains, only requesting the messages in any given chain that they are missing.
A user is essentially a keypair that can sign and encrypt messages. A node is an installation on a device with its own database. It's important that one user be able to use multiple nodes and keep them syncronized. The opposite may also be true, where one node hosts several users, but this is not currently a priority. All of a user's settings and internal state should be as portable, secure, and propagatable as any chain.
Public chains are simply chains that are not encrypted at all. Any node that recieves a public chain can read it, and if they have the user's public key, verify it.
Private chains are encrypted with another user's public key. Only this user can read and verify them.
Internal chains are encrypted symmetrically with the user's passphrase. They are used to store internal state in a form that can be replicated across the network.
Friend chains are encrypted symmetrically with a secret that has been distributed to a group of nodes. Generally, the secret distribution will go like this:
- Alice generates a random secret and adds it to Peter, Paul and Mary's private chains.
type: 'friend-chain:secret',
content: {
secret: String, // Secret that is used to encrypt messages in friend chain
chain_id: String, // ID of the friend chain in question
start: Number // Sequence of first message that is encrypted with this secret
}- She then encrypts messages with the secret before adding them to the friend chain.
- Peter, Paul and Mary replicate this message and index it with
['content.chain_id', 'content.start']- When decrypting a message in the friend chain, Peter, Paul and Mary retreive this message with the following query.
{
k: ['content.chain_id', 'content.start'],
v: [message.chain_id, [null, message.sequence]],
peek: 'last'
}'Unfriending' is also possible, by sending a new secret to everyone in the group except for the ex-friend.
Each node maintains a leveldb. Entries are indexed within the leveldb using level-librarian. For simplicity all messages are encoded as JSON.
All messages have the same metadata schema:
var message = {
previous: String, // Hash of the previous message in the chain
pub_key: String, // Public key of the author
chain_id: String, // ID of the chain this message belongs to
sequence: Number, // Ordinal sequence of this message in the chain
timestamp: Number, // Timestamp when the message was created
signature: String, // Signature of the rest of the message
type: String, // Optional - this is to prevent collisions in the `content` property
content: JSON // Used to store arbirary data
}The metadata is all properties on the message, except for content. This metadata is not encrypted. If encryption takes place, it is done to the content property of the message. All metadata is generated by sea-tale. There are also several types of messages with content generated by sea-tale. It is an important principle that all messages used internally by sea-tale can be generated by an API call. This way, the implementer never has to communicate with sea-tale by crafting special messages.
Entries are indexed using level-librarian. Index documents are not replicated between nodes. The user is also able to pass indexes for their own messages. (TODO write API)
To replicate, a node makes a request with a chain ID and an integer. The response is a stream of all messages with that chain ID and a sequence number higher than the integer. Since messages are encrypted, there is no effort to limit the messages that can be returned.
Each node maintains an internal chain of all the other chains that it is following. This way, it is able to request synchronization of the right chains.
type: 'core:follow',
content: {
chain_id: String
}Index
var index = 'type'Get list of chains that must be synchronized
{
k: 'type',
v: 'core:following'
}Get last in sequence of each chain
{
k: 'sequence',
peek: 'last'
}Replication request:
{
pub_key: String
chain_id: String,
latest: String
}pub_key, chain_id, and latest come from request
{
k: ['pub_key', 'chain_id', 'sequence'],
v: [pub_key, chain_id, [latest, null]]
}Send messages
Messages must be validated as they are saved.
In the interest of simplicity and profile portability, we are storing all state in internal chains. This includes the keypair. Better have a good password!
type: 'core:keys',
content: {
pub: String,
priv: String
}Index
var index = ['type', '$latest']Get keys
{
k: 'type',
v: 'core:keys'
}Users will want to use multiple devices. It is an important thing to have in a finished product. One way to enable multiple devices is with a device_id on each message. The same chain on multiple devices is then replicated as seperate chains, but viewed as one chain, interpolated with timestamps. This works ok for many use cases, like status messages and the like, but does not provide any strong guarantees around ordering of messages, and is probably not suitable for building data structures on top of.
For this reason, it might be better to leave out multi-device support at the core level, and operate off the assumption that one chain is only ever appended to by one device. A layer can then be built on top of this structure to provide various kinds of chain processing and combination, like the timestamp interpolation method detailed above.
It's important to allow one public key to be used across devices.
It would be simple to identify chains with a random id long enough to ensure global uniqueness. However, this is not strictly needed, as a chain only needs to be unique among other chains from the same device. If we use a random id, nodes will need to keep some sort of record outside the chain system.
How to deal with collisions etc in chain_ids?
GUID: Pros:
-
globally unique, collisions are not a concern
-
simplifies some aspects of replication, as the chain_id is the only info needed Cons:
-
modules looking for a chain will need to store some record of which chain is theirs
-
chain_ids are somewhat redundant, as they really only need to be unique within the namespace of a given user
- is this really the case? What about group chains etc? Need to come up with a really flexible id scheme
Probably least invasive to validate chains only when saving. For now, the whole chain must be present to validate. Later, the author of a chain could add a message authorizing partial validation (starting at a specific point).