Data structures in Swarm

Introduction

Fdfs (FairOS-dfs) can be used to store two kinds of data for a given user.

Personal user data /files

To store files and organize them in to logical partitions, Fdfs provides two ways

• One is to create Logical Drives called Pods akin to c:/ or d:/ in windows
• The seconds one is to create directories and arrange them in a nice order like any other file system

Data related to the applications that the user uses

For storing application related data, we need to have some kind of data structure to add/ update/remove information from the data. In this scope, there are two kinds of actors that needs to store information in Swarm.

1. Application data for all users

   • A KV store to have username <-> pod key mapping (where pod key is a specific user pod allotted for the application)
   • A search application which wants to store the global index and pointers to the original documents

2. Application specific data in the users pod which can be added/ updated/ removed / searched etc

   • A List to add elements as they are created / received ( Email inbox, Spotify playlist, Felfele Posts, reditt post etc)
   • A Key Value store (which does not preserve insertion order) but can be searched with key (To sore information about the applications)
   • A Key Value store, where Value is a Feed with Series i.e. which maintains all the version added to the value ( comments of a blog etc)
   • A Documents store (insert a json doc and index on specific fields to query)

Roadmap

• Two person DH secret exchange (to share files, pods to a single user)
• Extended Triple Diffie-Hellmann (X3DH) Key exchange for group secret exchange( to comment on some one else post, to add to a common Data structure)
• Feeds - Series ( to store a list of events like posts, emails etc)
• Feeds - Aggregate (for consolidating events like a distributed counter)
• One user KV store (file formats like levelDB)
• Multi user KV store (using trie with feeds as nodes)
• Documents indexes for KV stores