From f4e644f714ad5383f7c475987a69b4b3bc7e7186 Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Mon, 12 Feb 2018 10:44:53 +0000 Subject: [PATCH 01/11] Start entry invalidation RFC Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 content/entry-invalidation/index.md diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md new file mode 100644 index 0000000..90fa9ca --- /dev/null +++ b/content/entry-invalidation/index.md @@ -0,0 +1,27 @@ +--- +rfc: +start_date: 2018-02-12 +pr: +status: draft +--- + +# Entry Invalidation + +## Summary + +This RFC proposes a way to invalidate an entry. + + +## Motivation + +Currently, the append-only nature of Registers doesn't cope well with +mistakes. A situation that has arised more than once is getting two records +about the same thing with different identifiers because a human mistake. +Another situation is having a change in the history of a record that is wrong. + +We need a way to mark entries as invalid so we can keep compatibility with +other copies of the Register (consumers, replicas, etc). + +## Explanation + +TODO From 2f25040e8aeded324f469305b19b1730c50caffc Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Wed, 21 Feb 2018 11:53:41 +0000 Subject: [PATCH 02/11] Add entry sketch Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index 90fa9ca..112e073 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -24,4 +24,15 @@ other copies of the Register (consumers, replicas, etc). ## Explanation +The proposal is to introduce a new type of entry under the “user” type that +allows listing a set of entry identifiers [TODO: entry number or entry hash?]. +This mechanism would allow invalidating discrete entries and by extension +invalidating the full history for a key. + +### RSF + TODO + +``` +append-entry user ?? 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000 +``` From f529e0588e0b34fa2d47b169909a6fdf9319979a Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Wed, 21 Feb 2018 13:10:23 +0000 Subject: [PATCH 03/11] More sketching Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index 112e073..6778426 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -33,6 +33,29 @@ invalidating the full history for a key. TODO +By entry hash + +``` +append-entry user ?? 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000;sha-256:0000000000000000000000000000000000000000000000000000000000000001 +invalidate-entry user 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000;sha-256:0000000000000000000000000000000000000000000000000000000000000001 +``` + +Or by entry number + ``` -append-entry user ?? 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000 +append-entry user 2018-02-12T10:11:12Z 3;234;355 +invalidate-entry user 2018-02-12T10:11:12Z 3;234;355 ``` + + +### JSON + +```json +{ + "entry-number": "356", + "revoked-entries": ["3", "234", "355"], + "entry-timestamp": "2018-02-12T10:11:12Z" +} +``` + +TODO: It could be modelled as regular entries and special items. From d7ca4f0f46a6415383d14493e9fbfce499a5f2ac Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Thu, 22 Feb 2018 08:11:23 +0000 Subject: [PATCH 04/11] Add alternative approach Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 57 +++++++++++++++++++++++------ 1 file changed, 46 insertions(+), 11 deletions(-) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index 6778426..587857d 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -24,31 +24,32 @@ other copies of the Register (consumers, replicas, etc). ## Explanation -The proposal is to introduce a new type of entry under the “user” type that -allows listing a set of entry identifiers [TODO: entry number or entry hash?]. -This mechanism would allow invalidating discrete entries and by extension -invalidating the full history for a key. +The proposal is to introduce a new type of entry that allows listing a set of +entry identifiers [TODO: entry number or entry hash?]. This mechanism would +allow invalidating discrete entries and by extension invalidating the full +history for a key. -### RSF -TODO +### Special entry + +This approach defines a new type of entry. Caveat is that the shape of the +entry is incompatible with the existing one. + +#### RSF By entry hash ``` -append-entry user ?? 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000;sha-256:0000000000000000000000000000000000000000000000000000000000000001 invalidate-entry user 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000;sha-256:0000000000000000000000000000000000000000000000000000000000000001 ``` Or by entry number ``` -append-entry user 2018-02-12T10:11:12Z 3;234;355 invalidate-entry user 2018-02-12T10:11:12Z 3;234;355 ``` - -### JSON +#### JSON ```json { @@ -58,4 +59,38 @@ invalidate-entry user 2018-02-12T10:11:12Z 3;234;355 } ``` -TODO: It could be modelled as regular entries and special items. + +## Alternative approach: Special entry key "entries:revoked" + +This approach keeps entries uniform but claims a key as reserved. The +description of what entries are revoked is held in a kind of item with a +specific shape. + +[TODO: Review if "entries:revoked" is the right reserved key] + +#### RSF + +``` +append-entry user entries:revoked 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000 +``` + +#### JSON + +```json +# Entry 356 + +{ + "index-entry-number": "356", + "entry-number": "356", + "key": "entries:revoked", + "item-hash": [ "sha-256:0000000000000000000000000000000000000000000000000000000000000000"], + "entry-timestamp": "2018-02-12T10:11:12Z" +} + +# sha-256:0000000000000000000000000000000000000000000000000000000000000000 + +{ + "id": "entries:revoked", + "entry-numbers": ["3", "234", "355"] +} +``` From b05571948a829dce116347e83c16f67b0fcadf27 Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Thu, 22 Feb 2018 08:20:18 +0000 Subject: [PATCH 05/11] List of properties of the solution Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index 587857d..ba3dd83 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -60,7 +60,7 @@ invalidate-entry user 2018-02-12T10:11:12Z 3;234;355 ``` -## Alternative approach: Special entry key "entries:revoked" +## Alternative approach: Reserved entry key + new item shape This approach keeps entries uniform but claims a key as reserved. The description of what entries are revoked is held in a kind of item with a @@ -94,3 +94,15 @@ append-entry user entries:revoked 2018-02-12T10:11:12Z sha-256:00000000000000000 "entry-numbers": ["3", "234", "355"] } ``` + +#### Properties + +* Entries are consumed regularly (no breaking changes). +* Knowledge of the reserved key is required to filter it out when collecting + list data. +* Records don't list reserved keys. You don't get the same information if you + get it from `/records` or from `/entries`. Emphasises the dichotomy between + porcelain and plumbing. +* Revoked entries require a mechanism/documentation to explain how to apply + them when creating the list of records. +* Entry proofs are kept intact and invalidations are part of the tree. From fe661e17d91a9e9f00a616953892505f542d0377 Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Thu, 22 Feb 2018 09:20:45 +0000 Subject: [PATCH 06/11] Rename reserved key to chore:revoked-entries Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index ba3dd83..4d2c789 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -66,12 +66,12 @@ This approach keeps entries uniform but claims a key as reserved. The description of what entries are revoked is held in a kind of item with a specific shape. -[TODO: Review if "entries:revoked" is the right reserved key] +[TODO: Review if "chore:revoked-entries" is the right reserved key] #### RSF ``` -append-entry user entries:revoked 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000 +append-entry user chore:revoked-entries 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000 ``` #### JSON @@ -82,7 +82,7 @@ append-entry user entries:revoked 2018-02-12T10:11:12Z sha-256:00000000000000000 { "index-entry-number": "356", "entry-number": "356", - "key": "entries:revoked", + "key": "chore:revoked-entries", "item-hash": [ "sha-256:0000000000000000000000000000000000000000000000000000000000000000"], "entry-timestamp": "2018-02-12T10:11:12Z" } @@ -90,7 +90,7 @@ append-entry user entries:revoked 2018-02-12T10:11:12Z sha-256:00000000000000000 # sha-256:0000000000000000000000000000000000000000000000000000000000000000 { - "id": "entries:revoked", + "id": "chore:revoked-entries", "entry-numbers": ["3", "234", "355"] } ``` From 3db2d0d94bf4e200b37694a2966d19f81b25ba22 Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Thu, 22 Feb 2018 09:24:03 +0000 Subject: [PATCH 07/11] Add downside for alternative approach Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index 4d2c789..69bb6f8 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -68,6 +68,10 @@ specific shape. [TODO: Review if "chore:revoked-entries" is the right reserved key] +### Cons + +* This approach claims a key as reserved. + #### RSF ``` From eb926b1b0d648f27da2d88861c10f5936c703b7d Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Thu, 22 Feb 2018 10:18:50 +0000 Subject: [PATCH 08/11] Clarify alternative proposals Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 33 ++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index 69bb6f8..727487c 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -24,19 +24,25 @@ other copies of the Register (consumers, replicas, etc). ## Explanation +[TODO: Currently two alternatives are explored] + + +### Alternative A: Special entry + The proposal is to introduce a new type of entry that allows listing a set of -entry identifiers [TODO: entry number or entry hash?]. This mechanism would +entry identifiers. This mechanism would allow invalidating discrete entries and by extension invalidating the full history for a key. -### Special entry - This approach defines a new type of entry. Caveat is that the shape of the entry is incompatible with the existing one. +[TODO: entry number or entry hash?] + #### RSF + By entry hash ``` @@ -59,22 +65,33 @@ invalidate-entry user 2018-02-12T10:11:12Z 3;234;355 } ``` +#### Properties + +* New entry type without key. Breaking change diverging previous designs. +* Records are unaffected. +* Revoked entries require a mechanism/documentation to explain how to apply + them when creating the list of records. +* Entry proofs are kept intact and invalidations are part of the tree. + + + +### Alternative B: Reserved entry key + new item shape -## Alternative approach: Reserved entry key + new item shape +The proposal introduces a reserved key and a new type of item where entry +identifiers are listed. This mechanism allows invalidating discrete entries +and by extension invalidating the full history for a key. -This approach keeps entries uniform but claims a key as reserved. The -description of what entries are revoked is held in a kind of item with a -specific shape. [TODO: Review if "chore:revoked-entries" is the right reserved key] -### Cons +#### Cons * This approach claims a key as reserved. #### RSF ``` +add-item {"id":"chore:revoked-entries","entry-numbers":["3", "234", "355"]} append-entry user chore:revoked-entries 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000 ``` From 2936b80c7385ab93f0a3ff787cae41c65ec0cdfa Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Thu, 22 Feb 2018 12:17:33 +0000 Subject: [PATCH 09/11] Add CSV example Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 37 +++++++++++++++++++---------- 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index 727487c..fd4154a 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -9,7 +9,8 @@ status: draft ## Summary -This RFC proposes a way to invalidate an entry. +This RFC proposes a way to invalidate an entry so previous changes considered +mistakes can be removed from the computed list of records. ## Motivation @@ -20,7 +21,9 @@ about the same thing with different identifiers because a human mistake. Another situation is having a change in the history of a record that is wrong. We need a way to mark entries as invalid so we can keep compatibility with -other copies of the Register (consumers, replicas, etc). +other copies of the Register (consumers, replicas, etc) and allow computing +the correct(ed) list of records. + ## Explanation @@ -29,14 +32,10 @@ other copies of the Register (consumers, replicas, etc). ### Alternative A: Special entry -The proposal is to introduce a new type of entry that allows listing a set of -entry identifiers. This mechanism would -allow invalidating discrete entries and by extension invalidating the full -history for a key. - - -This approach defines a new type of entry. Caveat is that the shape of the -entry is incompatible with the existing one. +The proposal introduces a new type of entry that allows listing a set of entry +identifiers. This mechanism would allow invalidating discrete entries and by +extension invalidating the full history for a key. It also introduces a new +RSF command to avoid overloading the `append-entry` command. [TODO: entry number or entry hash?] @@ -67,7 +66,7 @@ invalidate-entry user 2018-02-12T10:11:12Z 3;234;355 #### Properties -* New entry type without key. Breaking change diverging previous designs. +* New entry type without key. Breaking change diverging from previous designs. * Records are unaffected. * Revoked entries require a mechanism/documentation to explain how to apply them when creating the list of records. @@ -91,7 +90,7 @@ and by extension invalidating the full history for a key. #### RSF ``` -add-item {"id":"chore:revoked-entries","entry-numbers":["3", "234", "355"]} +add-item {"id":"chore:revoked-entries","entry-numbers":["3","234","355"]} append-entry user chore:revoked-entries 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000 ``` @@ -116,6 +115,20 @@ append-entry user chore:revoked-entries 2018-02-12T10:11:12Z sha-256:00000000000 } ``` +#### CSV + +```csv +# Entry 356 + +index-entry-number,entry-number,entry-timestamp,key,item-hash +356,356,2018-02-12T10:11:12Z,chore:revoked-entries,"sha-256:0000000000000000000000000000000000000000000000000000000000000000" + +# sha-256:0000000000000000000000000000000000000000000000000000000000000000 + +id,entry-numbers +chore:revoked-entries,3;234;355; +``` + #### Properties * Entries are consumed regularly (no breaking changes). From 8f45f7198213a51fcd3bb5760b88279399aeeb1f Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Thu, 22 Feb 2018 15:13:20 +0000 Subject: [PATCH 10/11] Reformat examples Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index fd4154a..d5fd0ed 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -83,9 +83,6 @@ and by extension invalidating the full history for a key. [TODO: Review if "chore:revoked-entries" is the right reserved key] -#### Cons - -* This approach claims a key as reserved. #### RSF @@ -96,9 +93,9 @@ append-entry user chore:revoked-entries 2018-02-12T10:11:12Z sha-256:00000000000 #### JSON -```json -# Entry 356 +**Entry 356:** +```json { "index-entry-number": "356", "entry-number": "356", @@ -106,9 +103,11 @@ append-entry user chore:revoked-entries 2018-02-12T10:11:12Z sha-256:00000000000 "item-hash": [ "sha-256:0000000000000000000000000000000000000000000000000000000000000000"], "entry-timestamp": "2018-02-12T10:11:12Z" } +``` -# sha-256:0000000000000000000000000000000000000000000000000000000000000000 +**Item sha-256:0000000000000000000000000000000000000000000000000000000000000000:** +```json { "id": "chore:revoked-entries", "entry-numbers": ["3", "234", "355"] @@ -117,14 +116,16 @@ append-entry user chore:revoked-entries 2018-02-12T10:11:12Z sha-256:00000000000 #### CSV -```csv -# Entry 356 +**Entry 356:** +```csv index-entry-number,entry-number,entry-timestamp,key,item-hash 356,356,2018-02-12T10:11:12Z,chore:revoked-entries,"sha-256:0000000000000000000000000000000000000000000000000000000000000000" +``` -# sha-256:0000000000000000000000000000000000000000000000000000000000000000 +**Item sha-256:0000000000000000000000000000000000000000000000000000000000000000:** +```csv id,entry-numbers chore:revoked-entries,3;234;355; ``` @@ -132,6 +133,8 @@ chore:revoked-entries,3;234;355; #### Properties * Entries are consumed regularly (no breaking changes). +* This approach uses reserved key in the user space. Requires avoiding clashes + with user data. * Knowledge of the reserved key is required to filter it out when collecting list data. * Records don't list reserved keys. You don't get the same information if you @@ -140,3 +143,5 @@ chore:revoked-entries,3;234;355; * Revoked entries require a mechanism/documentation to explain how to apply them when creating the list of records. * Entry proofs are kept intact and invalidations are part of the tree. + + From b56b9167532d22c1a7bdd37b149d968eb0423d3c Mon Sep 17 00:00:00 2001 From: Arnau Siches Date: Thu, 1 Mar 2018 13:14:22 +0000 Subject: [PATCH 11/11] Tidy up proposal A Signed-off-by: Arnau Siches --- content/entry-invalidation/index.md | 22 +++++++--------------- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/content/entry-invalidation/index.md b/content/entry-invalidation/index.md index d5fd0ed..135cf58 100644 --- a/content/entry-invalidation/index.md +++ b/content/entry-invalidation/index.md @@ -16,7 +16,7 @@ mistakes can be removed from the computed list of records. ## Motivation Currently, the append-only nature of Registers doesn't cope well with -mistakes. A situation that has arised more than once is getting two records +mistakes. A situation that has arisen more than once is getting two records about the same thing with different identifiers because a human mistake. Another situation is having a change in the history of a record that is wrong. @@ -34,22 +34,15 @@ the correct(ed) list of records. The proposal introduces a new type of entry that allows listing a set of entry identifiers. This mechanism would allow invalidating discrete entries and by -extension invalidating the full history for a key. It also introduces a new -RSF command to avoid overloading the `append-entry` command. +extension invalidating the full history for a key when all entries for that +key are flagged as invalid. It also introduces a new RSF command to avoid +overloading the `append-entry` command. -[TODO: entry number or entry hash?] +[TODO: entry number or entry hash? Entry number is aligned with existing ways +to refer to entries. Entry hash invalidates the entry by its content] #### RSF - -By entry hash - -``` -invalidate-entry user 2018-02-12T10:11:12Z sha-256:0000000000000000000000000000000000000000000000000000000000000000;sha-256:0000000000000000000000000000000000000000000000000000000000000001 -``` - -Or by entry number - ``` invalidate-entry user 2018-02-12T10:11:12Z 3;234;355 ``` @@ -69,11 +62,10 @@ invalidate-entry user 2018-02-12T10:11:12Z 3;234;355 * New entry type without key. Breaking change diverging from previous designs. * Records are unaffected. * Revoked entries require a mechanism/documentation to explain how to apply - them when creating the list of records. + them when computing the list of records. * Entry proofs are kept intact and invalidations are part of the tree. - ### Alternative B: Reserved entry key + new item shape The proposal introduces a reserved key and a new type of item where entry