FIP: External cast data #171
Replies: 21 comments 91 replies
-
Hey @vrypan! I was going to write up a similar FIP along with Steve that suggested long casts be managed through an extension of the embeds array and uses IPFS protocol URIs. We'd like to throw our support behind this or something like it. Couple of questions for you that came up in our own internal discussions:
An alternative we talked through is less flexible but likely more performant: extend the embed array specifically for long casts and instead of pointing to a metadata file, you point directly to the long text content. Makes it as easy to render as it is to render images and videos today. Again, the big tradeoff is flexibility. Would love to hear your thoughts. |
Beta Was this translation helpful? Give feedback.
-
The metadata target is interesting. This could be done without a protocol change by following the frames playbook and just having a url include meta tag specifying |
Beta Was this translation helpful? Give feedback.
-
product UX questions:
|
Beta Was this translation helpful? Give feedback.
-
My initial thoughts:
I'd prefer it if we support clients to treat the content of the URL as an HTTP message with its own header (including Content-Type) and body section. All languages have ways to handle HTTP messages. For long casts particularly, clients could support This will be the simplest and most future proof approach IMO, after all, it's HTTP. The OG. |
Beta Was this translation helpful? Give feedback.
-
I like @deodad idea of doing something similar to what frames does. But can we expand this to support any kind of long files? It will depend on clients what to render and can potentially allow the creation of many interesting use cases. For instance: A user shares an embed link. It returns a set of defined meta tags:
I like @gabrielayuso idea of the So, let's say Warpcast wants to integrate long texts. A user starts typing, and once they reach the max characters, a message is displayed saying, "Your cast is longer than normal, Warpcast will show it but other clients may not display it properly." When the user is done typing, they cast, and the client should handle uploading that content to an external resource, getting the URI, and embedding it. Cooler clients can even use the allowed cast characters for mentions and other embeds. |
Beta Was this translation helpful? Give feedback.
-
(Updated FIP has been moved to the main post to make the discussion easier to follow) |
Beta Was this translation helpful? Give feedback.
-
If you think the FIP should also include guidelines for clients to present this type of content, please share your thoughts.
|
Beta Was this translation helpful? Give feedback.
-
FIP updated:
|
Beta Was this translation helpful? Give feedback.
-
Sharing code (https://warpcast.com/michaelsullivan/0xb94476e8) is a known problem. How about we extend this to somehow support code snippets? Just indicating that the text is pre-formatted would be easy for clients (just use a fixed width font), but allows sharing of code and... ASCII art :-) What do you think, @goksu? This would be a nice differentiator for Farcaster. |
Beta Was this translation helpful? Give feedback.
-
I updated the FIP. |
Beta Was this translation helpful? Give feedback.
-
What would happen if the external fetch fails? Every single client that's trying to access that particular long form text would fetch on the fly. I'm assuming that the people creating long text will probably put a CDN but for some who aren't and get a sudden spike in traffic will face the hug of death similar to Reddit early days. Having a graceful fallback in cases would help. Would need to think about CDN, caches and who would be doing it.
I don't think it makes sense for protocol to cache it. The other thing is updating it. What if the poster posts, then updates it? The latest post would need to get refetched and cache busted. |
Beta Was this translation helpful? Give feedback.
-
I like the idea very much.
I believe it would be useful to expand on the specification of
and so on. |
Beta Was this translation helpful? Give feedback.
-
I don't think having a prefix in the URL is a good way of signalling, and if every protocol extension go this way it is going to be painful. |
Beta Was this translation helpful? Give feedback.
-
I prefer this way: #136 It seems to me that adding JSON linked data to a webpage is a better approach. If you have just json data, you can have a proxy gateway url that takes that json and renders open graph image fallback, frame fallback and serves that JSON linked data. This means that app builders can have a blue check/green check fallback by falling back to the open graph without having to implement each content type. |
Beta Was this translation helpful? Give feedback.
-
I built a client (3 clients in 1 actually) to show how this FIP impacts, or doesn't, clients. The demo app showcases the option of just linkifying these embed URLs, natively supporting them, and doing nothing. I think it does a good job especially of highlighting that adoption of this FIP will not break existing clients in any way. |
Beta Was this translation helpful? Give feedback.
-
I do not believe we even have to add an extended metadata, link rendering or protocol specification. I do not believe it needs more standardization then to agree up on at which number of replies stringed together we call it a long cast or as it is called in most of social media today "thread". The easiest solution by farAdding a "show more" button if there is a reply to a cast from the cast author would result in similar UX to what other plattforms already have with long posts, but keep it a very simple client level UI change. If one clicks show more unroll the whole thread into a nice reader view and let it be read like a blog post with reader mode in current browsers e.g. Firefox This also works great backwards with existing threads that can now be read more easily and the second cast in a thread doesn't suffer the "I am not visible since no one shows replies on the top feed level before clicking on the first one" faith. Making the writing experience pleasent then is a change to the cast editor to allow overflows and simply render them as thread similar to what supercast.xyz is already doing. This introduces NO change and is backward compatible with anyone doing nothing or not supporting it since everyone renders casts and replies. A secondary viewIf there really "needs to" be a change then I'd love for a Telegram style "instant view" where the link is precached and presented to the user in reader mode. Then clients could allow writing by adding paragraph or other's editor directly in the client and post there via API and the cast is a regular one with embedded link to the public post. Then clients that want to support it render webpages not via opengraph but frame style rendering with instant view. This could also be prefetched or even cached so it opens instantly for the user without another http request by the client or the user when accessing it. If this takes of, then one can argue about adapting frame specification to add instant view and maybe even extend what can be instant viewed to include different kinds of URI schemes like IPFS, Arweave and others proposed in the discussion #171 ConclusionI firmly do not believe this requires any change and encourage any client to go ahead like others in this thread have done to implement, measure usage and come back with number showing if others need to do it to or not. |
Beta Was this translation helpful? Give feedback.
-
The newest iteration of this proposal makes a ton of sense. Flexible and backwards compatible. I have no further feedback. Nice work @vrypan! |
Beta Was this translation helpful? Give feedback.
-
Great work on this spec @vrypan and @davidfurlong and others. If I could provide some feedback, I would nudge us toward extending the fc OG tags as others like @porenes have suggested above. I'm actually agnostic towards what we use, as long as it's consistent with Frames. Clients already have to parse OG tags to handle frame support, and there's a lot of existing infrastructure around frames that could be relatively easily extended to support this (and other future types of casts). We have talked about using JSON for Frames instead of OG tags, and it's something we can migrate to in the future for both frames and longcasts. I think supporting two different types of metadata for "rich content" is a lot of complexity for the ecosystem. Concretely, I would propose the following: Add an e.g. <meta property ="fc:render" content ="longcast" />
<meta property ="fc:longcast:format" content="markdown" />
<meta property ="fc:longcast:content:url" content ="<ipfs or https link>" />
// or
<meta property ="fc:longcast:content:raw" content ="**Markdown Content**\nThis is an example long cast" /> If this is successful, we can look into moving Frames and Longcasts to start using JSON/JSON-LD down the line because it does have some benefits over meta tags. In that scenario, the page would have a single meta tag like <meta property ="fc:content" content="<url to json-ld document>" /> And the json file describes the frame/longcast/other future types. |
Beta Was this translation helpful? Give feedback.
-
I think this is fine. Basically switch the FIP to be about frames.
Which aligns with my request to support markdown frames
https://warpcast.com/gabrielayuso.eth/0x6d5b09cb <https://warpcast.com/gabrielayuso.eth/0x6d5b09cb>
…On Fri Jun 7, 2024, 12:26 AM GMT, Sanjay ***@***.***> wrote:
Great work on this spec @vrypan <https://github.com/vrypan> and @davidfurlong <https://github.com/davidfurlong> and others. If I could provide some feedback, I would nudge us toward extending the fc OG tags as others like @porenes <https://github.com/porenes> have suggested above.
I'm actually agnostic towards what we use, as long as it's consistent with Frames. Clients already have to parse OG tags to handle frame support, and there's a lot of existing infrastructure around frames that could be relatively easily extended to support this (and other future types of casts). We have talked about using JSON for Frames instead of OG tags, and it's something we can migrate to in the future for both frames and longcasts. I think supporting two different types of metadata for "rich content" is a lot of complexity for the ecosystem.
Concretely, I would propose the following:
Add an fc:render tag to the frames spec. Defaults to frame, but can be set to longcast (matches @type in your spec). And then use the existing frames conventions for describing the longcast
e.g.
<meta property ="fc:render" content ="longcast" /> <meta property ="fc:longcast:format" content ="markdown" /> <meta property ="fc:longcast:content:url" content ="ipfs or https link" /> // or <meta property ="fc:longcast:content:raw" content ="**Markdown Content**\nThis is an example long cast" />
If this is successful, we can look into moving Frames and Longcasts to start using JSON/JSON-LD down the line because it does have some benefits over meta tags.
In that scenario, the page would have a single meta tag like
<meta property ="fc:content" content ="<url to json-ld document>" />
And the json file describes the frame/longcast/other future types.
—
Reply to this email directly, view it on GitHub <#171 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAPN5JDL5AQKGYKBVAJZRTZGD44RAVCNFSM6AAAAABIFE6WBWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMOJVGE3TG>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
It seems people are generally aligned on splitting the FIP. The next question is: who will use this if we ship it? I hope someone like Paragraph will be excited about this. The reading experience is clearly superior to frames, which is what they use. A reason they might not want to is because frames let you "subscribe" to the author and this proposal has no equivalent. Is there some way to bake that into this standard to establish a link back to the author? If not I worry that paragraph will prefer to keep using frames and this feature might not take off. cc @seeARMS |
Beta Was this translation helpful? Give feedback.
-
Title: External cast data (long casts)
Type: Standards FIP
Authors: @vrypan, @davidfurlong
Abstract
This is a proposal to enable long casts. The long cast content is stored at an external URI. The proposal is designed to be easily extendable to support more features in the future, without hub modifications.
Problem
This FIP was triggered by the discussion on long casts (see https://warpcast.com/v/0xa9456e70).
In addition, a look at FIP ideas and Farcaster discussions indicates that many developers are interested in embedding a number of data types in casts: location, audio, events, etc. This FIP defines long casts in a way that will allow embedding various data types in the future with minimal changes.
Specification
Long casts are stored in external storage (not hubs) and are embedded to casts using
Embed.url
.Only plain text long casts are supported by this FIP.
Data Format
The content of a long cast is stored in JSON Linked Data format:
Embedding the payload
There are two ways to link to a long cast.
1. json-ld payload embedded in an HTML page.
The json-ld payload can be embedded in an HTML page:
and applications can use
Embed.url
to point to the HTML page URL.In this case, clients will have to fetch the HTML, parse it and check for a json-ld longcast payload, in the same way they check for Open Graph tags to display frames and page previews.
This is a backwards compatible way that will work nicely with web content shared on Farcaster (for example, blog posts can provide a longcast json-ld payload and clients can show the whole blog post as a long cast).
The disadvantage of this method is that clients have to load an HTML page without knowing in advance what to expect, and check for OG tags, then check for json-ld, and decide what to do depending on what the results.
2. Stand-alone json-ld payload
The json payload can also be hosted at a URL and
Embed.url
can link directly to it. Linking directly to the json payload is more efficient than embedding it in HTML.In this case, the
url
must be in the formatfc:<url>
that indicates to clients that the URL points to a json-ld payload.Examples:
fc:http://www.test.com/longcast.json
fc:ipfs://QmWMo7MrnpSAH2dMKNhcrvLeKdyyqjyrAdy9vUfEdyqcwU
Why use the
fc:
URI schema?fc:
URL will always point to json-ld data.fc:
, they would show a “no preview available” because they are not aware of the payload structure, now they will show the same because they do not know how to deal with thefc:
URI schema.schema=fc
,path=http://....
), and then treat thepath
as the new URL.Supported protocols
This FIP does not explicitly limit the transfer protocols clients are expected to support.
However, clients are encouraged to support at least HTTP, HTTPS and IPFS.
Presentation
If a cast includes both a body and a long cast, clients are expected to render both and I provide a visual indicator (for example a separator) that lets users know which part corresponds to each one. The cast body (if present) is expected to be rendered before any long cast content.
Size Limit
The JSON payload (either embedded or stand-alone) size must be less than 32kb.
This limit is set to help clients allocate resources and protect them from malicious payloads that could allocate an arbitrary memory size.
The limit also allows any content that could be published as a long tweet (limit 25,000 characters) to be posted as a long cast. (When using unicode characters this may not always be the case, but this feels like a good approximation.)
Wherever possible, clients are advised to reject the json data structure when its size is more than 32kb.
Compatibility
Long casts embedded in an HTML page provide multiple ways (OG tags, HTML tags) that allow clients that do not support this FIP to show an alternative representation of the long cast (for example, a frame or an abstract).
Long casts directly embedded as stand-alone json will appear as unsupported link types in clients that do not support this FIP. Given the early stage of the Farcaster ecosystem, this is considered an acceptable tradeoff to provide a future-proof, clean and efficient way to embed rich data in casts.
Extensibility
Clients that implement this FIP, can easily support additional json-ld payload schemas if they decide to do so in the future.
schema.org has a number of schemas (including events, places, products, reviews and other) already supported by major services like Google, but new ones can also be defined.
JSON-LD is available in a number of popular programming environments. A comprehensive list can be found at json-ld.org.
Hub requirements
No changes.
Storage requirements
No change in storage requirements.
Additional thoughts
The combination of long casts stored in decentralized storage, combined with permissionless channels can provide a truly decentralized content publishing platform that requires NO web servers: the web3 version of the web2 blog.
Revisions:
fc:
schema to link directly to payloads.structured+
tofc+
. The term feels more natural given the context, and easier to use.text/pre
is also part of the spec. It indicates pre-formatted text such as code or ASCII art.Beta Was this translation helpful? Give feedback.
All reactions