FIP: Location in user profiles

[aditiharini]

Title: Location in user profiles
Type: Implementation FIP
Authors: @aditiharini, @sanjayprabhu, @christopher, @vrypan, @varunsrin

Abstract

Add a new User Data type that lets users share location data publicly.

Problem

If users opt-in to share location, clients can build features that:

• Offer recommendations based on a user's location (e.g. farcaster meetup in your city today)
• Notify people when someone they follow is visiting (e.g. Alice is in New York today)
• Find people with certain skills in your area (e.g. developers in France)

Warpcast has implemented some location features at a client level, showing demand for this. We're proposing a standard to publish this data to the user's profile.

Specification

Our requirements for the location data type are:

• Strong typing, so that other clients can process it
• Rough precision, so that users' don't accidentally share the location of their home
• Erasable, so that users could redact a previously published location from the protocol

Encoding

Location values as encoded as latitude/longitude pairs rounded down to two decimal places. For example, the value for the city of Los Angeles would be 34.05, -118.24. A 2dp rounded lat/long provides an approximate precision of 11 square kilometers around the point, which prevents someone from publishing sensitive location data by accident. These values can easily be converted to and from other representations like geohashes or google's place id for use in applications.

Message Format

The UserDataType is extended to support an eighth value for USER_DATA_LOCATION. All User Data messages with this type must also pass the following validation rules:

1. The value must match the regular expression geo:-?[0-90]\.[0-9]{2},-?[0-180]\.[0-9]{2} which is compliant with the geo URI standard. Latitude and longitude must be specified with exactly 2 decimal place precision.
2. The maximum size of the value is 18 bytes.

Example: geo:-34.56,123.45

Rationale

Why not use an unstructured string?

A string with a city description is very difficult to use because of lack of standardization of location names (e.g. New York vs New York City) and people's tendency to use different levels of precision (e.g. New York vs Brooklyn)

Why are the latitude/longitude values rounded?

A user or application may accidentally publish a sensitive location like someone's home using unrounded values. Since Farcaster is a public protocol this data may end up indexed in databases and become impossible to erase. Rounding ensures that location data is not traceable to a specific building.

Why not use geohashing instead of latitude/longitude values?

Geohashing has a couple of benefits.

A geohash's main benefit is database indexing. Locations that are close together with have lexicographically similar hashes which is useful if you are processing very high volumes of location-based queries. While this might be useful in some applications, it's quite trivial to convert a lat/long into a geohash format if necessary.

Second, the precision of a geohash is encoded in the representation. You can easily identify the precision via the length of the geohash string. Latitude, longitude values can be represented as strings, integers (given fixed precision), or floating point numbers and validating that the values have a certain level of precision requires more complicated logic (as compared to a string length check). The complexity here is localized and taken on by hubs rather than applications.

The main downside of a geohash is that developers will almost certainly need to convert a geohash into a latitude, longitude pair before using their preferred location service as most support lat/long out of the gate and not geohash. Doing this requires understanding the algorithm and picking a library to do the translation, which is a considerable effort. Some geohash variants have limited library support and are pretty complicated to understand.

It's also hard to pick a single geohash-style algorithm to use in the protocol as there are multiple standards with different tradeoffs. The ideal algorithm choice depends on how the data is being used.

It is simpler and most ergonomic from a develop perspective to just publish values as lat/long and let developers convert them into a specific geohash format if necessary.

Why not represent the latitude/longitude values using JSON or protobuf?

• JSON is expensive space-wise and doesn't buy us much. We can use JSON parsing libraries, but parsing a string with lat/long manually isn't that complicated.
• Encoding in protobuf is space efficient(saves 50%) but only if we encode the location values as integers. If we encode the location data as integers, all clients need to translate between integers and fixed precision decimals. Encoding in protobuf also requires extending the UserDataBody message in a custom way just for location data since all other user data is represented as a string. Ultimately, it's unclear that the space vs ergonomics tradeoff is worth it here and we will migrate to a more space efficient location later if this feature is widely adopted and the space savings seem worth it.

Appendix

Comparing results of 1dp vs 2dp rounding

The results show that 1dp rounding often results in translating one locality to some nearby locality, even for pretty major cities. Accepting this would be at the detriment of the UX of the location features within and/or across apps.

Converting Lat/Long coordinates to a Place ID

const googleClient = new GoogleClient({});
const resultTypes = [
  PlaceType2.locality,
  PlaceType2.administrative_area_level_3,
  PlaceType2.administrative_area_level_4,
  PlaceType2.sublocality,
  PlaceType2.sublocality_level_1,
];

const parseLatLongFromProtocol = async (latlng: [number, number] // truncated lat, long values from a hub) => {
 // Contains place id, human-readable description, etc. for use by the application
  return await googleClient.reverseGeocode({
    params: {
      latlng,
      result_type: resultTypes,
      key: GOOGLE_PLACES_API_KEY,
    },
  });
};

const latLongToPublish = async (placeId : string // collected by the app via user-input, gps, etc) => {
  const placeDetails = await googleClient.placeDetails({ params: { place_id: placeId, key: GOOGLE_PLACES_API_KEY } });
  return truncateTo2Dp(placeDetails.data.result.geometry.location)
}

[vrypan]

Is this a proposal to add a new UserDataType? Regardless of format, how will the location information be added to the protocol?

[vrypan]

I like the idea.

What is the reason for ossifying this in the protocol? Who needs it, what is the problem it solves? Why location and not something else, for example URL, or language? Or why not a URL pointing to a json that could provide additional info in a structured way?

[goksu]

There are many things that can be moved to protocol, sure. But this property is already shown on Warpcast profiles and has been asked by developers to be available already.

My personal mental model is anything visible on user profiles on common social apps like X, Instagram etc. will probably make its way to the user data on Farcaster protocol.

[vrypan]

I won't insist more, I mostly do it because I feel that every FIP should come with a problem statement.

Regarding the implementation: Do we want to keep adding special UserDataTypes whenever there is a need for something more? Why not use a more flexible type that is a pointer to a json whose spec can be easily extended without protocol changes that affect the hubs? More freedom may even help new use cases bubble up, that we have not considered.

Again, I'm not against adding a new location UserDataType. Just asking the questions that should have an answer :-)

[davidfurlong]

mostly do it because I feel that every FIP should come with a problem statement

@vrypan for any sort of distance based app (events, marketplace, recommendations, social) the approximate geographic distance to the +- 5km is a prerequisite before you can do anything.
Farcaster protocol data is accessible in frames, so after this FIP is merged, frames or third party farcaster apps could offer location specific recommendations of events, products, other users and other social interactions without having to ask the user for their location

[aditiharini]

@vrypan Yes, this will be a new USER_DATA_TYPE.

Thanks for pushing on the problem statement. Location has been supported on Warpcast for a long time and we've received positive feedback on location-based features. Given the location feature has reached maturity and there's demand for it, It's a good candidate for pushing into the protocol.

[rishavmukherji]

Personally I like the idea of Open Location Code as user data type

[goksu]

Any special characteristics of OLC that pushes you on this direction instead of lat / long?

[shashank-reddy-code]

thanks for the proposal. this will unlock some new consumer use cases.

geohash does have an advantage with better support in elastic and postgres (through ST_GeoHash)

question: is the location going to be verified by clients?

[aditiharini]

One thing to note here is that clients who would like to use geohash in db can easily translate a lat/long into a geohash prior to storage. Latitude, longitude is supported universally across different APIs and gives clients maximum flexibility to use and store the data as they wish.

We do not plan for location data to be verified. Are there reasons that verification is particularly appealing for location data?

[andrei0x309]

Users must be able to delete any location data anytime they desire otherwise it's not privacy-friendly and not GDPR compliant.

Warpcast already takes location without consent, and it leaks it openly example link:

https://warpcast.com/~/locations/ChIJT608vzr5sUARKKacfOMyBqw

It doesn't matter if is not pinpointed exactly.

STATE-CITY is still regarded as PI under EU law.

[vrypan]

It's not "leaking" data. It is broadcasting the data users submitted to be broadcasted. What you link to is a list of profiles that wanted their location to be displayed on their profile. It's private information until you make it public yourself.

[andrei0x309]

No, you're wrong my location is empty, on warpcast.com, there's no way to delete it from warpcast DB, and I never consented to give it out.

For example here is an account made on using chain and has country 'XX', because a phone not was used:

But in the one where a phone was used, the location was stored without my consent.

I mean is not a big deal I know USA companies have a poor practice of handling personal data.

But especially if is on protocol user must have access to delete it anytime otherwise will violate EU law, that was my main point.

So in the EU even if someone consents for some PI to be made public you also have the right to delete that data anytime, especially if that data is designated as PI, and anything location-related is designated as such.

[varunsrin]

There are two different things that you are conflating here:

1. Public Location - When you set your location on Warpcast, you opt in to sharing a city-level location publicly. This can be removed at any time from your profile. This proposal is to move this data into the protocol, and you can delete it any time.

2. User Location - Warpast also keeps track of your country-level location (fetched by Cloudflare) for compliance with US laws. This is only visible to Warpcast and the user themselves. This is permitted by Article 17 of GDPR.

[andrei0x309]

You're right in the sense of conflating points, compliance can't be determined without an internal audit anyway, and also I don't care if it is compliant or not. The internet is full of providers that aren't.

What I care about is in case the data will exist at the protocol level, the right to be forgotten to be extended there, that's much more important than how Warpcast handles PI data because the exposure level is wider, my main point is that user should be in control of deleting data that can be classified as PI.

[varunsrin]

Yes, we'll always ensure that you can delete location data from the protocol.

[lyoshenka]

Geospatial hash seems right to me. Here's my thinking:

• description strings are too loose. they're human-readable but not dev-usable
• lat/lng and geohashes are roughly equivalent (you can convert from one to the other) but with geohashes you can decide how coarse or fine you want the resolution to be, which supports privacy and is open to future resolution changes. the tradeoff is its somewhat harder for devs, but thats a one-time fixed cost so it's worth it.

[andrei0x309]

IMO the best type is a variable type because if any client introduces the data, data can't be verified anyway, and can be spoofed so data will always be true-thy but not truthful.

With a variable type, a user can have any kind of defined location data type.

From a privacy standpoint, the only requirement is to have a delete mechanism at protocol that doesn't leave traces(like delete messages that can be found) and to be opt-in, and not mandatory.

Personally, I don't see many applications for location data, especially because if you have multiple clients is impossible to trust location, but I guess for some people there might be some limited applications even if data can't be trusted.

[ryan-foamspace]

agreed

[varunsrin]

With a variable type, a user can have any kind of defined location data type.

this would be annoying for developers - imagine building a feature that lets you find users in a location near you. you now have to translate every possible type into a single type so that you can run geo-distance calculations.

it would be much better to standardize on a single type that meets our requirements. i'm supportive of adding the flexibility to change types in the future by having a type modifier in the value, but i see no good reason to encourage multiple types today.

[davidfurlong]

+1 for Geospatial hash

How important is the proximity detection feature?

for any sort of distance based app (events, marketplace, recommendations, social) the approximate geographic distance to the +- 1km seems pretty important, to the degree that a city name is not good enough. Additionally as a developer having to reverse geocode city names/addresses to coordinates to calculate this is a real pain (and slow).

OLC seems significantly simpler than H3. H3 seems fairly complex and optimized for running geo-algorithms like shortest path + optimization problems (but you can always convert from OLC -> H3 to do these if needed).

Another note: as a dev, you'll probably need to compute a human readable label for each geospatial hash to display in the UI. For very precise hashes you can use the neighborhood or city, for less precise ones you may have multiple cities to choose from, in which case you perhaps choose the county or the region - but then the data has more accurate data on the user's location than is displayed to the user, which may be not expected by the user. The supported degrees of precision may be useful to restrict, or alternatively support a human readable label that is stored alongside the geospatial hash for the convenience of developers, as well as the convenience for the user to be able to override it if imprecise. This would also guarantee the user has a consistent human readable location label across apps

[ryan-foamspace]

thanks, good point. and in general in favor of this FIP and nice to see geohashes considered

[davidfurlong]

Lat + Lng with protocol-limited number of decimal places would effectively guarantee some radius of privacy similar to hashes

[davidfurlong]

@davidfurlong What is the value of representing the uncertainty radius in the protocol? How do you anticipate the applications will use this information?

Here's an example.

One privacy conscious user sets "Los Angeles" as their location, and it's encoded via Lat, Lng at the arbitrary center of los angeles 34.0549,-118.2426 (in downtown), but they actually live next to Erewhon in Venice, at 33.9897, -118.4618. Another non privacy conscious user lives in downtown LA and sets their location as such, as a location of 34.0543,-118.2431.

You're building an app that let's people meet up for lunch with another. You want to match users that are within 20 minutes of eachother. You search user's coordinates to calculate the distance, and if less than 20 minutes, you match the users for lunch. These two users are, by coordinates within 5 minutes of eachother, so they're automatically matched. However in reality they're more than an hour in traffic from each other. The app has no good way to know that the first user has chosen a more privacy preserving location precision and provides a subpar experience as a result. If the app had known, it would have asked for a more precise location just for it's app (not public & permanent), or excluded this user from automatic matches.

This example isn't made up - my previous startup was a volunteering marketplace and we experienced this problem

[vrypan]

We shouldn't try to make this something that covers every possible use case of location data. If someone wants to create the Foursquare-caster, they will have to build their own app, and collect location data in a way that fits them. Same for business locations (where you actually need addresses), and so on.

Based on the use cases described in the FIP, the problem definition is to put city-level location on the protocol. If we think this is not broad enough, we should revise it. If it is, it seems that low accuracy lat/long is a good solution.

[vrypan]

Alternatively, we can say that it's not the protocol's role to set this type of restrictions and it's up to the apps and the users who pick them, to do decide based on their specific needs, balancing accuracy and privacy accordingly. (Which I also find a valid approach)

[ryan-foamspace]

We have done some work with Geohashes and Crypto-spatial coordinates, check it out
https://f-o-a-m.github.io/foam.developer/tutorials/intro_to_csc.html

[aditiharini]

Interesting! Why did you choose to represent location via geohash over other options?

[ryan-foamspace]

We need a location standard for blockchain applications that is free, open source and interoperable so that we can securely connect offline spaces to online assets.

Another benefit of the geohash standard is that it is in the public domain. This doesn’t mean that the geohash doesn’t have limitations. The CSC is a 96 bit identifier, defined as the first 12 bytes of the Keccak-256 hash of the concatenation of the geohash with the ethereum address. In Solidity, this looks like this:
function computeCSC(bytes12 geohash, address addr) returns(bytes12){
return bytes12(sha3(geohash, addr));
}

[aditiharini]

Did you consider a truncated/rounded latitude, longitude value? That's the option we're weighing geohashing against so would be useful to understand how you weighed these 2 options.

[ryan-foamspace]

if precision is not of concern truncated can work, but EVM does not play nice with floating points re lat.long see this chart

[varunsrin]

the main benefit of geohashing seems to be that they are good for indexing since locations that are close together will have similar hash values. this isn't really a big consideration for the protocol and even warpcast which uses location data today doesn't use this feature.

the main downside is that its a little more complicated than lat/long - there are multiple standards, there may not be great libraries or support for each standard in every framework.

so i'm tempted to lean lat/long for its simplicity and because you can convert a lat/long into any kind of geohash if you want to use it for your own indexing purposes.

[vrypan]

According to this, https://github.com/google/open-location-code/blob/main/Documentation/Reference/comparison.adoc#geohash-36

Geohash-36 is based on a 6x6 grid where each cell is identified by a character. This causes codes over a boundary to be dissimilar even though they may be neighbours

[lyoshenka]

rounded lat/lng makes sense to me as long as the the protocol enforces precision (by e.g. limiting how many bits can be used), which helps avoid client mistakes and makes clear to all clients what the precision is.

[zachterrell57]

This looks great! I wrote about this a little here:
#165

I think just adding location to User Data is a good start - hopefully we can eventually see it extended to casts and even as parent URLs.

Tactically, one thing I think we should think about is the degree of precision we're allowing. Like you said, with one decimal we get ~11km of precision. While I think that degree of precision is useful for some apps ("X user is in your city!"), I think this limits a lot of use cases. For reference, the the entire city of San Francisco is about 11km wide (from ocean to bay), so any client wanting to display regional hotspots in the city would not be able to due to the lack of granularity.

Based off this chart: I think the protocol should allow for up to 3-4 decimal places of precision, and it should be up to the client to decide what degree gets published. I think most clients will go with the ~11km you suggest, but if there was a special feature a client was trying to produce I don't think they should be limited by the protocol

[christopherwxyz]

been thinking about extending location to the cast since Dev Day. short term we could author an FIP to expand the embed type to lat/long and possibly add a dedicated lat/long slot for a location embed. it wouldn't break backwards compatibility.

long term i would like to see better metadata attached to casts.

[varunsrin]

I'm in favor of extending embeds with a location standard

[vrypan]

Nothing's stoping anyone from using https://en.wikipedia.org/wiki/Geo_URI_scheme in embeds, right?

[christopherwxyz]

we would need a hub validation to ensure a valid geo URI:

1. is a valid URL
2. is a valid lat/long
3. non-valid URIs are thrown and handled

geo:foo,bar is a valid geo URI, but not a valid lat/long:

try {
  const url = new URL('geo:foo,bar');
  console.log('Valid URL:', url.href);
} catch (error) {
  console.log('Invalid URL:', error.message);
}

invalid geo URI also makes it to the hub today, but is ignored within Warpcast:

https://explorer.neynar.com/0xd1c17bfeda99561136ea92e014b5f9775efb45b4
https://warpcast.com/christopher/0xd1c17bfe

[christopherwxyz]

added a new FIP to continue the conversation of geo URI as a valid Embed type: #197

[christopherwxyz]

i really like the idea of permissionless user data location. it's a good first pass at more user-centric interop and less transaction or content-centric interop. i would like to see place id at some point for user location "status" in the near term, but we don't have a permissionless model over claiming a place.

i also think that privacy is fundamentally important. self-governing down to 1 decimal point balances public data and privacy preferences. 👍

[andrei0x309]

Same, I think privacy comes from self-governing, control, as long as the user can add/delete/update anytime it doesn't matter how precise is the location, it will not infringe on privacy if the user owns the control over it, there might be an exceptional case where the user loses the control over the account, some law might still force you to be able to get rid of that data.

That's an important issue if the network can have a method to remove it in exceptional cases, without user approval.

[varunsrin]

it doesn't matter how precise is the location

im more worried about accidents. imagine someone writes an app to publish location and the user accidentally logs their home location. someone else writes an app to archive all location changes on the hubs for well known users and makes it public. then users can end up in a situation where their family's home location is permanently online.

[vrypan]

Regarding the message format.

I'm a bit skeptical about asking hubs to do regex matching to decide if a message is valid or not. If I was building an alt hub in Go or Rust or C, this would probably mean importing a library I did not need before. Maybe it's not a big deal right now, but do we want to add such dependency deep in the protocol? Also, regex matching is expensive. How could this affect hubs (or even sequencers) at 100x scale? Is it worth the cost?

I would prefer if the regular expression was a soft requirement, imposed by clients: "Sure, you can put anything there that's X bytes long, but 99% of the clients will ignore it, if it does not match this regex".

[aditiharini]

I assume the regex matching you're referring to here is for the purpose of detecting the precision of the provided lat/long data on the hub side.

Couple thoughts:

1. I don't expect user data updates to be that frequent so I'm not very concerned with the performance impact
2. I'm not sure we need to pull in a library to do regex matching to detect the dp precision. I could imagine multiplying by 10 and detecting whether the number is an int. It looks like different languages have different builtins for this sort of thing (e.g. isInteger in JS, .fract() = 0 for rust).

[vrypan]

Sure, I was referring to

The value after the prefix must match the regular expression ____.

Regarding 1, yes, and no. :-) It's good not to design something based on specific user patterns. If validating a USER_DATA_LOCATION delta needs 10x the CPU than other deltas because it does a regex match, it means that bot accounts maybe be able to bring down hubs by mass broadcasting this type of deltas -or a bug that makes a client spawn thousands of these deltas.

But if it's designed in such a way that validation can be done with standard math, that's great, and my concerns are irrelevant.

In this direction, I would go with +034.1-118.0, ie.

• integer part padded to 3 digits,
• decimal part one digit (even if it's zero),
• always including +/-,
• no prefix (not sure of how the prefix will be used, what is it?).

This is something easy to parse, you know exactly where to find the values in the string, no if/then/elses, standard length of 12 bytes (or even remove the dots, and go down to 10 bytes, snd stick to integer arithmetic).

[varunsrin]

1. I don't think it will be that expensive - you don't actually need a full blown regex parser, you can implement the rules using simple bit checking. I did something like this for the username restrictions in the solidity contract, and if it can be done in that environment, it can be done easily in hubs.

2. If I'm wrong, it will be trivial to ship a backwards compatible protocol change that relaxes the requirement when we reach 100x scale.

I'd rather optimize for all developers being able to use the data and have a better location experience today.

[aditiharini]

We explored the 1dp precision and found that it lacks sufficient precision to provide a good user experience. We're going to use 2dp precision because it's still sufficiently privacy preserving (accurate only to 1 km^2) and provides for a much better user experience.

Here's some data we collected:

We ran some selection of locations through Google's APIs. We started with place ids, translated to the specific lat/long values, then rounded to 1dp/2dp, and translated back to place id to see if we got the same value we started with. With 1dp precision, we'd often end up moving the location over to a nearby locality. What this means in practice is that either (1) apps will be out of sync with respect to location because the originating app will have a precise location and others will infer some nearby but not same location based off the rounded lat/long. (2) the originating app will need to validate that the user is only shown a location based off the rounded values. This means if a user input Brooklyn, NY, the app would need to round to 1dp and show only Queens, NY or NYC. Inputting Brooklyn, NY and not seeing Brooklyn, NY on your app is pretty sad. You'd have the same experience for Paris, France which is a major city and not even a region inside a city.

With 2dp precision, there are still some cases where rounding results in us getting no results or inaccurate results, but it happens way less frequently and only for really small localities. We will need to accept some form of this problem with nearly any level of rounding, but 2dp precision strikes a reasonable balance in the tradeoff between user experience and privacy preservation.

[aditiharini]

Quick update:
We have some hub stability issues that we're focusing on addressing right now. This will be our top priority til resolved (hopefully within a few days). We will finalize this spec and pick up work on decentralizing location immediately after! If you have any further thoughts, feel free to drop here.

[vrypan]

This is a very interesting use case, where location data in the user profile can be useful.

https://warpcast.com/j-valeska/0xdcf5d3cf

trying to convert a datetime to locale string in a frame using user location, is it even possible?

[pcowgill]

FYI the title is incorrect