Merge pull request #139 from ietf-wg-jose/serialization-improvements

Serialization improvements

Author: dwaite

draft-ietf-jose-json-web-proof.md

@@ -425,28 +425,40 @@ The algorithm is responsible for representing selective disclosure of payloads i
 
 Each disclosed payload MUST be base64url encoded when preparing it to be serialized.  The headers and proof are also individually base64url encoded.
 
-Like JWS, JWP supports both a Compact Serialization and a JSON Serialization.
+Like JWS, JWP supports both a Compact Serialization and a JSON Serialization. These serializations both represent the same JSON-based header, payload and proof, and are thus interchangeable without breaking the proof value.
 
 ## Compact Serialization {#CompactSerialization}
 
-The individually encoded payloads are concatenated with the `~` character to form an ordered delimited array. Any non-disclosed payloads are left blank, resulting in sequential `~~` characters such that all payload positions are preserved.
+The compact serialiation provides a space-efficient encoding of a JWP in URL-safe characters. In addition to the alphabet of unpadded BASE64 URL-safe encoding, it uses the "." and "~" characters as separators.
 
-A payload which is disclosed but which contains no data (i.e. a zero-length octet string) is encoded as a single `_` character of data, which is not a valid result from base64url-encoding a value.
+All binary data is BASE64URL encoded, including the octets of the UTF-8 encoded headers and the individual payloads and proof values.
 
-Additionally, an algorithm MAY supply multiple octet strings for a proof. These are concatenated with the `~` character to form an ordered delimited array.
+Payloads and proofs are each concatenated into a single text form by concatenating the BASE64URL encoded values using the `~` character.
 
-The headers, concatenated payloads, and proof value are then concatenated with a `.` character to form the final compact serialization.  The issued form will only contain one header and always have three `.` separated parts.  The presented form will always have four `.` separated parts, the issued header, followed by the protected header, then the payloads and the proof.
+Individual payloads are allowed to be omitted; if a payload is omitted, it is represented as a zero-length text value, potentially resulting in leading, trailing, or consecutive `~` characters in the concatenated form.
+
+If a payload or proof value was a zero-length octet string, it does not get output as its zero-length BASE64URL-encoded form but as a single `_` character. This character does not represent a valid BASE64URL-encoded octet string, allowing it to be distinguished from normally encoded data.
+
+The issued form is created by concatenating the base64url-encoded issuer protected header, concatenated payloads, and concatenated proof separated each by a `.` character. The concatenated payloads MAY be omitted if the application is using detached payloads.
+
+The presented form is created by concatenating the base64url-encoded presenter protected header, base64url-encoded issuer protected header, concatenated payloads, and concatenated proof separated each by a `.` character. The concatenated payloads MAY be omitted if the application is using detached payloads.
 
 <{{./fixtures/build/bbs-holder.compact.jwp.wrapped}}>
 Figure: Compact Serialization of Presentation
 
 ## JSON Serialization {#JSONSerialization}
 
-Non-disclosed payloads in the JSON serialization are represented with a `null` value in the `payloads` array. A zero-length payload is represented as a zero-length base64url encoded sequence, the empty string `""`.
+The JSON Serialization is in the form of a JSON object, with property names representing the various components.
+
+The `issuer` key has a string value holding the BASE64URL-encoded issuer protected header. This key MUST be included.
+
+The `presentation` key has a string value holding the BASE64URL-encoded presentation protected header. It MUST be included for presented form, and MUST be omitted for issued form.
 
-Proofs are represented as an array of one or more encoded octet strings.
+The `payloads` key has an array value, representing the ordered sequence of payloads. If a payload has been omitted, it is represented by the JSON value `null`. A payload is otherwise reprented by the BASE64URL-encoded form of the payload octets. A zero-length payload does not have special encoding rules as needed by compact encoding, and is represented by the zero-length string output by BASE64URL. This key MUST be included unless the application is using detached payloads.
 
-This example flattened JSON serialization shows the presentation form with both the issuer and presentation headers, and with the first and third payloads hidden.
+The `proofs` key has an array value, representing the array of octet strings produced by the chosen algorithm. These octets are BASE64URL encoded into a JSON array.
+
+This example JSON serialization shows the presentation form with both the issuer and presentation headers, and with the first and third payloads hidden.
 
 <{{./fixtures/build/bbs-holder.json.jwp.wrapped}}>
 Figure: JSON Serialization of Presentation
@@ -479,6 +491,14 @@ unless the application knows that the encrypted content is
 a JWP by another means or convention,
 in which case the `cty` value MAY be omitted.
 
+# Detached Payloads
+
+In some contexts, it is useful to make statements about payloads which are not themselves contained within the JWP, similar to "Detached Content" in JWS [@RFC7515].
+
+For this purpose, the compact, JSON and CBOR serializations allow for all payloads to be omitted from a serialized form. While this is a legal serialization, it is not on its own able to be verified.
+
+The recipient is expected to perform some sequence of steps defined by the application to recreate the array of payloads, including order and optionality. This effectively recreates the fully specified serialization of the JWP.
+
 # Security Considerations {#SecurityConsiderations}
 
 Notes to be expanded:
@@ -798,6 +818,8 @@ for his valuable contributions to this specification.
 
   * Update reference to new repository home.
   * Fixed #83: Added encrypted JWPs.
+  * Added additional clarification around the compact and JSON serializations
+  * Added text around fully detached payloads
 
  -05