User cache storage design considerations

User cache storage design considerations

Server side

For the builtin implementation, we use a MongoDB collection to store our data.

Should we store the user_id in the metadata

In User-cache-data-format-design-considerations, we describe metadata and data structures for different types of entries. However, on the server, we store data for multiple users. How do we separate the caches by user id?

Options:

1. Store the uuid in the metadata. This is a fairly obvious solution.

• Pro: it is consistent to have all the metadata in one place.
• Con: since the UUID cannot be serialized, we have to strip it out before we return it.

2. Store the uuid as a top level field, at the same data as the metadata and data, as shown below. In this case, we would just retrieve the metadata and data separately while returning the values. { 'user_id': uuid, 'metadata': {}, 'data': {} }

• Pro: No need to strip anything out
• Con: Need to assemble result before returning it

3. Add the metadata and data under a new entry parent. The uuid is at the same level as the entry. { 'user_id': uuid, 'entry': { 'metadata': {}, 'data': {} } }

• Pro: No need to strip or reassemble anything
• Con: Queries need to be nested one more level. Looks ugly.

4. Separate databases for each user. Create at user creation. Delete at user deletion (deletion not yet implemented).

• Pro: More consistent with long-term vision. Although do we need physical, versus logical, separation?
• Cons: Mongodb collection scalability? Queries across users are difficult in the absence of a distributed query architecture

Decision###

We are going with option 2 because option 4 is too much of a change and is riskier, while option 3 adds a new field which may not exist in the long term, and which complicates queries unnecessarily.

Client side

Options:

1. Use a single sqlite table that expands the metadata but stores the data as a CLOB.

2. Create one table for each key, but with the same set of columns. This is basically the same as the prior option, only with a physical instead of a logical separation between the various tables. Note that it is possible to generate table "views" from the previous option that are identical to this option, simply by including the key in the select query.

3. Create one table for each key, but with different sets of columns for each table. This would include the metadata columns, but require configuration of which additional columns to load. At this point, this does not satisfy any known use case, and increases complexity, so we will defer implementing it until it is truly necessary.

Decision###

So we are going with the first option for now.