General Policy Review #32
Comments
Looks good
I like MetaList because it's like MetaEvidence, whose name was also a mistake. It's minor, will think about it.
Now I'm thinking both statements are confusing. Maybe just "the inclusion of an item is disputed", without mentioning the Editions at all. Since, the policy explains this later anyway.
The item considered registered is never mentioned in the policy because:
MetaList contains a
No idea how to do it cleanly
originally i just wanted to make the policy less verbose. but this self reference was not used much anyway, so it makes sense to remove this variable name.
the General Policy
I thought it was an easy way to convey the point of a heirarchy of policies the arbitrator should consider.
Fair simplification. I'm not sure though. But putting an optional suggestion for policy building is not appropriate for the General Policy, so it can leave.
Well, it's a may so you can skip it. but yeah I can just let it go.
Challenger has to provide the reason in the method challengeItem. The reason why it's provided immediately, is to avoid frontrunning the edits and figuring out why they're wrong later.
No, I think the exact reason should be provided ("exact" taken in a reasonable meaning). Reason + evidence. I will amend this.
Parallel challenges are not possible. Yes you can self challenge.
No, opposite. arbitrator just rules to keep in that case, no need to read the List Policy.
"Property" sounds too generic and I rather use tech or make up words. That removes semantical collisions. "Prop" is shorter than "Property of the Item".
Fair
No real reason. I'm not sure about this one, since field is very generic, probably people would use it for the Props and generate confusing language. Will gather opinions
Not sure how this works anymore. (context for other participants, we had a chat on telegram before I replied). Maybe the liveliness guarantees of the rollup allow to make this work. Tldr: the "frontrunning protection" of the general policy didn't take into account that a tx being public does not guarantee it gets into the block. I guess a terrible way to solve it is, do calls through a contract that will revert if it doesn't occur before the designated block. (bad ux in multiple levels), but it doesn't save editors.
Yes, only the last edition, under normal circumstances, is challenged.
done
well, it's an optional feature that the subgraph is supporting, so it may warrant be at the general policy. it was wrong from the beginning but reason was wrong, so Keep. challenger can put the reason now if he gets the next challenge. |
Not clear enough for a policy IMO.
It's actually worded as "may only" which is a totally different meaning from "may". "you may only do x" means that all alternatives to x are prohibited. So in this case, whether that was your intent or not, you were actually prohibiting the use of the other RFC2119 words.
I don't get the problem here. What frontrunning can happen? Either the "edit" is wrong and the challenger can justify why or it's not and they've just played themselves by "frontrunning" a valid entry. So "frontrunning" blindly is a losing strategy on average (unless the vast majority of submitted entries are invalid). If you just give one day to provide a reason during the evidence period, as long as the reason doesn't change passed one day, it's equivalent, except for the fact that the submitter has 1 day less to submit their own evidence.
I don't get what your objection is. I only ask that "the reason must be reasonably specific".
Then, your lists lose a lot of utility. Consider an NFT registry for instance. If I'm an NFT marketplace, I might want to only allow Kleros-certified NFTs to be sold. But if a minter can self-challenge with a bad reason, their NFT will exceed the defaultAgeForInclusion and be considered certified despite having not gone through the proper verification process.
Prop already has semantic collisions, but suit yourself.
I don't think "generic" names are an issue. Just define them in the definitions if you're afraid they might be misunderstood.
So, I looked at the code and I now see that only an item's "owner" can edit it. That makes things a lot simpler indeed. In that case, my suggestion is to drop the block-by-block requirement and replace it with a 1-hour delay (although you might want to make that configurable in the ListMeta/MetaList): "The challenged edition's IPFS URI must be recorded as part of the initial challenge reason (in the appropriate json field (for instance)) and must have been the active edition at the start of at least one block in the hour preceding the confirmation of the challenge transaction (including the challenge transaction's own block). More precisely, the following must hold true: C ≤ E + 3600 where C is the challenge block's timestamp in seconds and E is the timestamp in seconds of the latest block preceding or at C at the start of which the challenged edition was the active edition." |
RFC2119 "may" means optional, and RFC2119 is in place for the General Policy as well
my naive thought on edits was that, when you are editing an item it is likely that there's something wrong with it, on the context of list the version it was launched with, and you want to fix it. if this list version is the latest one, then the item is challengeable. because (in the version of the General Policy you reviewed) edit + challenge on the same block means you consider the edition available at the previous block, if it was challengeable, when you see the edit coming, you may want to challenge immediately to get the opportunity of finding the reason after the fact, then you can frontrun the challenge and figure out why its wrong later. you dont want this to happen, because you want item owners to be able to fix their wrong item instead of punishing them for it. if this wasn't added, then item owners and frontrun challengers have to play an arms race in this dark forest: owners edit their items which are correct both in the previous and new editions, and challengers have to figure out if the edit is a bait / enhancement, or a fix. i would also be fine for that. but the item owners would have more incentives for baiting if the challenge had to put a stake to be awarded to the owner if "Keep" is ruled. unfortunately, this is not the case in the contract. but, maybe it should be.
I misread, sorry.
when I was writing that response I also thought about this edge case, and I wrote an idea to handle it but then removed to not clutter the response. challenged items are not canonically included, either. for lists in which this creates an incentive to wrongly challenge to temporally remove items for griefing, ways to mitigate the damage are allowing duplicates and raising the amounts at stake. again, here it would be interesting to allow lists to force challengers to put skin in the game, because right now they only pay the arbitration fee and item owners get nothing out of it.
doesn't this allow a challenger to listen to editions taking place, and challenge an item to use a previous wrong edit?
|
You're right that's a perverse incentive not to self-correct mistakes, but on the other hand it means there was a mistake in the first place which should have been challenged. So although not optimal, this does not break the registry. |
I think pretty much all the other "challengeable" smart contracts have a challenger deposit (in addition to the juror fees that is). It's usually set to 0, but configurable.
Raising the amount required to challenge is one mitigation (and only a mitigation) and needs to be supported by the smart contract. Perhaps a better way to go about this would be an ad hoc appeal mechanism of sorts, where the number of jurors is increased *2+1 on each new challenge, either compared to the previous challenge first round, or compared to the last round in the Arbitrator. Both options have nasty edge cases though. At the end of the day, I feel like you're trying to solve two problems at once (stake curate and the challenge problem) while proposing a half-baked solution (or non-solution) to the second one. Perhaps it would be better to just go with the status-quo regarding challenge mechanics for now. At a minimum, I don't think this should be enforced by the General Policy, and there should at the very least be the option to override this in the List Policies. |
I will add this feature first, then I'll apply all this feedback onto the new policy if you want to review it then. I'll ping you by then. On the frontrunning mechanism, I thought about two completely different ways to make a good compromise:
I'm leaning towards 2nd option because it seems simple (even if it's offloading the complexity to a general policy). The default frontend and injected evidence will certainly help with this so it should not be an issue. The bad thing about using 2nd option is, (getting dystopian) people can run AIs and figure it out in a short amount of time, and we go back to the dark forest issue. But, tbh this can also happen in a block by block basis to it should just be embraced. It's not perfect but status quo could work
I think a very adamant way to fight it is to allow duplicates in the lists. The attacker would have to spend more and more resources to dunk the items, and the submitter gets more and more funds from the wrongful challenges. Keeping disputes in parallel would be difficult (the structure cannot afford to have an extra ~32-56 bits to keep a counter of disputes without an extra slot). I don't see how to make it work without keeping track of the counter. This is due to locked amounts, etc. Also, one thing: the challenges, if successful, can only remove the item. A dispute that rules to keep the item does not mean you cannot remove it with a new dispute, just making that clear. The item is always collaterized. |
b. "MetaList": Should be named "ListMeta" instead, or better still perhaps, "ListMetadata".
c. "Challenge: process in which the inclusion of an Item, in regards to an Edition, is put into question, and it creates a Dispute." If I've understood correctly, I suggest the following instead which should be a bit clearer: "Process by which an edition of an item is disputed. If the challenge succeeds, the Item is removed." (but is it the Item or the Edition?)
0'. You don't define when an item is to be considered registered. From what you've said previously I'm guessing this is up to the downstream dapps to decide but this is quite confusing to me.
0''. Instead of giving an unstructured list of definitions, I think an explanatory paragraph or two (where you can bold the keywords) would be more helpful.
"This document will refer to the General Policy as Policy from this point onwards." => "as the Policy" but you could remove this passage and just refer to it as "The General Policy".
"This document will refer to the General Policy as Policy from this point onwards. This also refers to all documents this Policy holds authority over." Second sentence is not clear at all. What's "this", and how can a document hold authority over other documents? Maybe you simply meant: "The combination of this document (the General Policy) and the List Policy of the specific list under consideration (as defined in the previous paragraph) will be referred to as the Policy (bold)."
Nitpick: You can refactor this: "The terms must, must not, shall, shall not, should, should not, may, required, recommended, not recommended and optional, have their meaning specified in RFC2119.
To ease understanding, the Policy may only use the following subset of terms: must, must not, should, should not and may."
=> "The terms must, must not, should, should not and may have their meaning specified in RFC2119. This applies to List Policies as well.".
3'. You say "the Policy may only use the following subset of terms" but what if I create a List Policy that uses other terms? Surely we don't want to consider it void in that case. Better not to insist on that point IMO.
"Upon creation of the Dispute, the Challenger must give a reason for removing the Item. It must be a valid reason according to the Policy. If this reason is not valid, the Arbitrator must rule to Keep the Item. This is the case even if the Item does not belong for any other reason, even if implied otherwise on the following clauses."
a. Upon is vague. Must the full reason be given in the same transaction as the one creating the dispute? Personally, I think it's good to give the challenger some time to formulate their reasoning after challenging. Otherwise, if the reasoning is complex, this could lead to multiple challengers spending time and effort writing their justification in parallel while only the fastest one will get to reap the benefits. I suggest you allow 24 hours to submit a justification.
b. "The reason should be clear and exhaustive. The reason must not be ambiguous or open to interpretation." I would add: "The reason must be reasonably specific (not e.g. "The policy does not abide by one of the rules.", or "The item is a duplicate of another item in the list." (without specifying the duplicate item))".
c. Are parallel challenges possible? I don't see how they could be. Which means one can easily slip an item in by self-challenging with an invalid reason. Or perhaps an item is not considered to be part of the list until it has been unchallenged for a sufficient amount of time, but in that case, the item can be attacked so it never makes the list. I personally dislike the current challenge format as well, from a juror's perspective especially, since, in case no reason is given or the reason given is bad, jurors have to check every single detail of the submission, but there's no simple way around it.
"Required Props" If you mean properties, I'd say "properties" in full. Prop already has several very different meanings in English.
"An Item must be removed if it lacks Props whose Column is marked as required." => "if it lacks a Prop/Property"
Why "column" instead of "field"? I'm also confused by the data structures here. You say a Prop is a (label, value) pair. Do you mean (field name, field value) pair then? If so, I'd drop the Prop/Column language altogether. E.g. "Required Fields" "An Item must be removed if it lacks a field marked as required", "Intrusive Fields" (maybe "Unspecified Field" would be a better name) "An item must be removed if it contains a field unspecified in the List Policy" (not sure if that was your intended meaning).
Regarding "Frontrunning, baiting and Editions", I'm sure you've thought about it, but why not have the edition be part of the txdata? In practice, being allowed to push a new edition once the challenge transaction is in the mempool will definitely allow attacks. And relying on flashbots is not acceptable since this service is not guaranteed to remain accessible and otherwise relies on trust assumptions.
I'm also confused: what happens if multiple editions are published in quick succession? Can only the last one be challenged? What happens to the previous ones? If the last edition is removed, is the item itself removed?
8'. TODO after you answer 8..
8''. Disregarding previous points: "the previous block in which the Challenge was created" ("in which" somewhat confusing) => "the block immediately preceding the one in which the Challenge was created (henceforth Previous Block and Challenge Block respectively)"
The text was updated successfully, but these errors were encountered: