Use body field as media caption (#1731)

* Use `body` field as media caption As per MSC2530. Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr> * Add changelog Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr> * Use `s` HTML tag in example Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr> * Move changed-in annotation Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr> --------- Signed-off-by: Kévin Commaille <zecakeh@tedomum.fr>
matrix-org · Feb 26, 2024 · 4cfe2fb · 4cfe2fb
1 parent 9a1f0ad
commit 4cfe2fb
Show file tree

Hide file tree

Showing 6 changed files with 140 additions and 8 deletions.
diff --git a/changelogs/client_server/newsfragments/1731.feature b/changelogs/client_server/newsfragments/1731.feature
@@ -0,0 +1 @@
+Use the `body` field as media caption, as per [MSC2530](https://github.com/matrix-org/matrix-spec-proposals/pull/2530).
diff --git a/content/client-server-api/modules/instant_messaging.md b/content/client-server-api/modules/instant_messaging.md
@@ -27,10 +27,18 @@ instead.
 
 Some message types support HTML in the event content that clients should
 prefer to display if available. Currently `m.text`, `m.emote`, `m.notice`,
-and `m.key.verification.request` support an additional `format` parameter of
-`org.matrix.custom.html`. When this field is present, a `formatted_body`
-with the HTML must be provided. The plain text version of the HTML
-should be provided in the `body`.
+`m.image`, `m.file`, `m.audio`, `m.video` and `m.key.verification.request`
+support an additional `format` parameter of `org.matrix.custom.html`. When this
+field is present, a `formatted_body` with the HTML must be provided. The plain
+text version of the HTML should be provided in the `body`.
+
+{{% boxes/note %}}
+{{% changed-in v="1.10" %}}
+In previous versions of the specification, the `format` and `formatted` fields
+were limited to `m.text`, `m.emote`, `m.notice`, and
+`m.key.verification.request`. This list is expanded to include `m.image`,
+`m.file`, `m.audio` and `m.video` for [media captions](#media-captions).
+{{% /boxes/note %}}
 
 Clients should limit the HTML they render to avoid Cross-Site Scripting,
 HTML injection, and similar attacks. The strongly suggested set of HTML
@@ -320,6 +328,49 @@ to the media repository, then reference the `mxc://` URI in a markdown-style lin
 Clients SHOULD render spoilers differently with some sort of disclosure. For example, the
 client could blur the actual text and ask the user to click on it for it to be revealed.
 
+##### Media captions
+
+{{% added-in v="1.10" %}}
+
+Media messages, comprised of `m.image`, `m.file`, `m.audio` and `m.video`, can
+include a caption to convey additional information about the media.
+
+To send captions, clients MUST use the `filename` and the `body`, and optionally
+the `formatted_body` with the `org.matrix.custom.html` format, described above.
+
+If the `filename` is present, and its value is different than `body`, then
+`body` is considered to be a caption, otherwise `body` is a filename. `format`
+and `formatted_body` are only used for captions.
+
+{{% boxes/note %}}
+In previous versions of the specification, `body` was usually used to set the
+filename of the uploaded file, and `filename` was only present on `m.file` with
+the same purpose.
+{{% /boxes/note %}}
+
+An example of a media message with a caption is:
+
+```json
+{
+    "msgtype": "m.image",
+    "url": "mxc://example.org/abc123",
+    "filename": "dog.jpg",
+    "body": "this is a ~~cat~~ picture :3",
+    "format": "org.matrix.custom.html",
+    "formatted_body": "this is a <s>cat</s> picture :3",
+    "info": {
+        "w": 479,
+        "h": 640,
+        "mimetype": "image/jpeg",
+        "size": 27253
+    },
+    "m.mentions": {}
+}
+```
+
+Clients MUST render the caption alongside the media and SHOULD prefer its
+formatted representation.
+
 #### Server behaviour
 
 Homeservers SHOULD reject `m.room.message` events which don't have a

diff --git a/data/event-schemas/schema/m.room.message$m.audio.yaml b/data/event-schemas/schema/m.room.message$m.audio.yaml
@@ -6,8 +6,29 @@ properties:
   content:
     properties:
       body:
-        description: "A description of the audio e.g. 'Bee Gees - Stayin' Alive', or some kind of content description for accessibility e.g. 'audio attachment'."
+        description: |-
+          If `filename` is not set or the value of both properties are
+          identical, this is the filename of the original upload. Otherwise,
+          this is a caption for the audio.
+        type: string
+        x-changedInMatrixVersion:
+          "1.10": This property can act as a caption for the audio.
+      format:
+        description: |-
+          The format used in the `formatted_body`. Currently only
+          `org.matrix.custom.html` is supported.
+        type: string
+        x-addedInMatrixVersion: "1.10"
+      formatted_body:
+        description: |-
+          The formatted version of the `body`, when it acts as a caption. This
+          is required if `format` is specified.
+        type: string
+        x-addedInMatrixVersion: "1.10"
+      filename:
+        description: The original filename of the uploaded file.
         type: string
+        x-addedInMatrixVersion: "1.10"
       info:
         description: Metadata for the audio clip referred to in `url`.
         properties:

diff --git a/data/event-schemas/schema/m.room.message$m.file.yaml b/data/event-schemas/schema/m.room.message$m.file.yaml
@@ -6,8 +6,25 @@ properties:
   content:
     properties:
       body:
-        description: A human-readable description of the file. This is recommended to be the filename of the original upload.
+        description: |-
+          If `filename` is not set or the value of both properties are
+          identical, this is the filename of the original upload. Otherwise,
+          this is a caption for the file.
+        type: string
+        x-changedInMatrixVersion:
+          "1.10": This property can act as a caption for the file.
+      format:
+        description: |-
+          The format used in the `formatted_body`. Currently only
+          `org.matrix.custom.html` is supported.
+        type: string
+        x-addedInMatrixVersion: "1.10"
+      formatted_body:
+        description: |-
+          The formatted version of the `body`, when it acts as a caption. This
+          is required if `format` is specified.
         type: string
+        x-addedInMatrixVersion: "1.10"
       filename:
         description: The original filename of the uploaded file.
         type: string

diff --git a/data/event-schemas/schema/m.room.message$m.image.yaml b/data/event-schemas/schema/m.room.message$m.image.yaml
@@ -6,8 +6,29 @@ properties:
   content:
     properties:
       body:
-        description: "A textual representation of the image. This could be the alt text of the image, the filename of the image, or some kind of content description for accessibility e.g. 'image attachment'."
+        description: |-
+          If `filename` is not set or the value of both properties are
+          identical, this is the filename of the original upload. Otherwise,
+          this is a caption for the image.
+        type: string
+        x-changedInMatrixVersion:
+          "1.10": This property can act as a caption for the image.
+      format:
+        description: |-
+          The format used in the `formatted_body`. Currently only
+          `org.matrix.custom.html` is supported.
+        type: string
+        x-addedInMatrixVersion: "1.10"
+      formatted_body:
+        description: |-
+          The formatted version of the `body`, when it acts as a caption. This
+          is required if `format` is specified.
+        type: string
+        x-addedInMatrixVersion: "1.10"
+      filename:
+        description: The original filename of the uploaded file.
         type: string
+        x-addedInMatrixVersion: "1.10"
       info:
         allOf:
           - $ref: core-event-schema/msgtype_infos/image_info.yaml

diff --git a/data/event-schemas/schema/m.room.message$m.video.yaml b/data/event-schemas/schema/m.room.message$m.video.yaml
@@ -6,8 +6,29 @@ properties:
   content:
     properties:
       body:
-        description: "A description of the video e.g. 'Gangnam style', or some kind of content description for accessibility e.g. 'video attachment'."
+        description: |-
+          If `filename` is not set or the value of both properties are
+          identical, this is the filename of the original upload. Otherwise,
+          this is a caption for the video.
+        type: string
+        x-changedInMatrixVersion:
+          "1.10": This property can act as a caption for the video.
+      format:
+        description: |-
+          The format used in the `formatted_body`. Currently only
+          `org.matrix.custom.html` is supported.
+        type: string
+        x-addedInMatrixVersion: "1.10"
+      formatted_body:
+        description: |-
+          The formatted version of the `body`, when it acts as a caption. This
+          is required if `format` is specified.
+        type: string
+        x-addedInMatrixVersion: "1.10"
+      filename:
+        description: The original filename of the uploaded file.
         type: string
+        x-addedInMatrixVersion: "1.10"
       info:
         description: Metadata about the video clip referred to in `url`.
         properties:
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Use the `body` field as media caption, as per [MSC2530](https://github.com/matrix-org/matrix-spec-proposals/pull/2530).