FIP-14: Residual Storage

[varunsrin]

Title: Residual Storage
Type: Implementation FIP
Authors: @deodad, @horsefacts, @sanjayprabhu, @varunsrin, @vrypan

Abstract

Hubs will preserve some messages when an account's storage expires for a better user experience.

Problem

Hubs revoke all messages when an account's storage expires and passes the 30-day grace period. Anyone viewed the account's profile or any recent threads it participated in will see missing data. If the account holder decides to renew the account, they'll also have to restore their profile (UserData) and follow (Links) messages from a backup or enter this in again.

Our storage design aggressively pruned messages because we weren't sure the Hub's storage model would scale. But now that we have several months of data, we can change the pruning logic for a better user experience.

Specification

A Farcaster account which has no storage is allowed the following limits:

CRDT Type	Message Limit
UserData CRDT	50 messages
Cast CRDT	50 messages
Reaction CRDT	50 messages
Verification CRDT	25 messages
Link CRDT	250 messages
UsernameProof CRDT	5 messages

If all these messages are fully used, they will consume ~ 100 kb of storage. But based on today's usage patterns, we expect them to be closer to ~ 60kb on average.

These limits will likely be changed and/or reduced as hub storage requirements increase with more users joining the network.

Hub API Changes

For a user with expired storage, GetCurrentStorageLimitsByFid must return:

{
    "limits":
    [
        {
            "storeType": 1,
            "name": "CASTS",
            "limit": 50,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 2,
            "name": "LINKS",
            "limit": 250,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 3,
            "name": "REACTIONS",
            "limit": 50,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 4,
            "name": "USER_DATA",
            "limit": 10,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 6,
            "name": "USERNAME_PROOFS",
            "limit": 1,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 5,
            "name": "VERIFICATIONS",
            "limit": 10,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        }
    ],
    "units": 0
}

Rationale

Will this cause hubs to run out of storage?

Residual storage increases the amount of data stored on hubs. In the short term (next year), we have reasonable confidence that this won't impose a significant burden on hub runners. In the long run, we expect we will have to revisit and update these limits.

Why is the limit for links so much higher?

An account renewing storage will find its feed empty if it has no links, and will have to do a lot of manual work to restore its social graph and get a useful feed. Casts and reactions, on the other hand, matter much less to the usability of the application. Keeping around a small number ensures recent conversations in other peoples feeds are preserved, but beyond that are much less important to the user experience.

Release

• The changes are backwards compatible and will go out in the next patch release.
• They will formally be part of the 2024.02.21 release.

Appendix A: Message Distribution on Hubs

The distribution of messages in each significant CRDT as of Jan 18th, 2024 are:

Data Type	Min Size	Max Size	Median Size	Average Size
Casts	149	845	179	275.06
Links	148	161	155	153.78
Reactions	153	238	171	171.36
Verifications	163	403	267	261.34
UserData	143	401	159	170.68

The distribution of message counts in each significant CRDT set as of Jan 18th, 2024 are:

Data Type	Min Count	Max Count	Avg Count	Median Count	p95 Count	p99 Count
Casts	1	18,808	77.87	7	237	1,236
Links	1	8,024	86.8	54	247	793
Reactions	1	26,014	151.34	10	553	2,680
Verifications	1	23	1.22	1	2	4
UserData	1	5	3.7	4	4	4

[sanjayprabhu]

Implementation farcasterxyz/hub-monorepo#1626

[vrypan]

Suggestion: Residual storage should be read-only.

I.e. messages by FIDs that are in residual storage mode are rejected. If not, I may think I can still post reactions and not realize that I only have 25 reactions.

[vrypan]

1. That's fair. Suggestion:
   a) if all storage units have expired, new messages are rejected. (I know it may add complexity in other places, just suggesting.)
   b) when pruning leave the "residual storage" messages (25 casts, 25 reactions, etc.) intact.

2. Not sure if this is right, but I have no facts-based counterargument. However, not being able to update my profile mentioning my old job or old project would have been a good reason to renew.

[varunsrin]

a) if all storage units have expired, new messages are rejected. (I know it may add complexity in other places, just suggesting.)

could add a timestamp based limit, which wouldn't be too bad and would encourage people to renew.

[sanjayprabhu]

That's an interesting approach. To do this, we'd have to track the timestamp at which the last unit of storage expired and reject all messages after that timestamp. We do have this data, so won't be too difficult to implement. We could also add this later down the line when units are actually about to expire.

[vrypan]

What if for now we just set the timestamp limit: When units expire, the account goes to read-only mode.

1. This makes users life easier: I'm on vacations for a month (August is a month that many people may be away), my units expire but that's ok, when I come back I try to post and realize I have to purchase units.

2. Content remains intact. In the future, when the ecosystem is more mature, we can have an FIP to deal with it. Maybe introduce storage units with longer expiration date. Maybe prune messages of expired accounts after two years of inactivity. Maybe set up a way for the community to undertake the cost of preserving some accounts. Maybe people that used FC early and then forgot about it they will come back because FC is much better now. I don't know.

How much would this actually impact hubs? Considering that storage cost always goes down, this could even be a strategy that makes sense even in the long term. I just feel that deleting content at this point in time will do more harm to Farcaster than good.

[vrypan]

... or maybe we realize that users generating 99% of the content actually renew, and it's ok. Or maybe in December 2024, fc has a big boom and goes 100x in a month and these old users that haven't logged in for months re-activate their accounts.

We have not been at this point before (units expiring), developers are not aware of the implications or the opportunities, users are not familiar with the impact -most of us, not even the impact of reaching storage limits. It would be wise to leave some space for errors and experimentation. It's hard to reverse the decision to delete content, but very easy to decide to delete it later if this turns up to be the right decision.

[vrypan]

What is the expected response when calling GetCurrentStorageLimitsByFid in this case? I propose zeroes.

[varunsrin]

cc @sanjayprabhu

[sanjayprabhu]

The response will look like this:

{
    "limits":
    [
        {
            "storeType": 1,
            "name": "CASTS",
            "limit": 50,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 2,
            "name": "LINKS",
            "limit": 250,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 3,
            "name": "REACTIONS",
            "limit": 50,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 4,
            "name": "USER_DATA",
            "limit": 10,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 6,
            "name": "USERNAME_PROOFS",
            "limit": 1,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        },
        {
            "storeType": 5,
            "name": "VERIFICATIONS",
            "limit": 10,
            "used": 0,
            "earliestTimestamp": 0,
            "earliestHash":
            {}
        }
    ],
    "units": 0
}

The units which represent the units of storage the user has available will be 0, and the limit for each store type will reflect the actual residual limit.

[varunsrin]

@sanjayprabhu and I were discussing limits, and it might be desirable to have a high follow limit, so you preserve your social graph. it's probably the hardest thing to recreate if you accidentally let your storage lapse, and while we have plenty of storage space on the hubs we should try to preserve as much of this as possible.

[vrypan]

I can see an opportunity here, for a service to proactively backup all state before storage units expire and let users restore it for a fee :-)

[andrewhong5297]

this would be a dope way to have a "free" trial for users to signup with, that expires in maybe 30 days. Would make onboarding to farcaster much easier/cheaper, making it more approachable for developers to build into their apps. I'm personally looking to build an embedded wallet + farcaster account creation flow in my site, but sponsoring $3 at the start with no user history is pretty costly/prohibitive.