METADATA_BLOCK_PICTURE in OGG

[oyla12]

Moin

I am trying to add image data to a stream. I have a FLAC/OGG stream that is actually fed by an ALSA source. Therefore I want to insert the image into the stream via harbor via http.

The general structure is as follows

# On Local device
input_stream = mksafe(input.alsa())
output.icecast(%ogg(%flac(samplerate=48000, channels=2, compression=5, bits_per_sample=16)), [...])

# On Remote VM
input_stream = input.harbor()
output.icecast(%opus(bitrate=192, channels=2, signal="music", application="audio", vbr="none", samplerate=48000), [...])
output.icecast(%vorbis.abr(bitrate=256, channels=2))
[...]

The OGG container provides the metadata_block_picture to store images. I had considered that it would probably be wise to simply feed in a file with an included image in meta_block_picture directly via the playlist as a test. If the image is passed through directly, then at least it would be clear that it is not filtered during processing, so a happy path to start from modifications.

I have therefore fed an image into an ogg/vorbis with the following line. The line is adapted from here, as an image I had a small 20kb .jpg. : https://superuser.com/questions/1708793/add-art-cover-in-ogg-audio-file

ffmpeg -i test.ogg -acodec copy -map 0:a -metadata:s:a METADATA_BLOCK_PICTURE=“$(cat vorbis.head test.jpg | base64 --wrap 0)” test_bild.ogg

The VLC player displays the image correctly, so it seems (with a little optimism) to be validly inserted.

I then injected the file and used playlist() to play a playlist with only this file as in my earlier tests. Playback works, but the image does not come through. I have two theories.

1. Theory: Either the image cannot be read correctly and the file is corrupt. The fact that I get an ffmpeg warning from the liqudisoap log could speak in favor of this:

Feb 02 00:06:53 overgangspunkt liquidsoap[84309]: 2025/02/02 00:06:53 [playlist_cover_m3u:3] Prepared "/etc/liquidsoap/testfiles/test_bild.ogg" (RID 6).
Feb 02 00:06:53 overgangspunkt liquidsoap[84309]: [mjpeg @ 0x7f3027d1e0] unable to decode APP fields: Invalid data found when processing input
Feb 02 00:06:53 overgangspunkt liquidsoap[84309]: [mjpeg @ 0x7f3027d1e0] No JPEG data found in image
Feb 02 00:06:53 overgangspunkt liquidsoap[84309]: [ogg @ 0x7f301b8ea0] Could not find codec parameters for stream 1 (Video: mjpeg, none(bt470bg/unknown/unknown), 8x14 [SAR 300:300 DAR 4:7]): unspecified pixel format
Feb 02 00:06:53 overgangspunkt liquidsoap[84309]: Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options

This is still too strange for me as an error message, so I checked another way. To limit the problem, I looked at what tooling is available and installed the “easytag” software. It offers a GUI to edit metadata and also to save the image to the file.

When you add a picture to an .ogg file with the software, it adds the picture to the meta_block_picture as expected. I also realized, that the METADATA_BLOCK_PICTURE is not shown using ffprobe, so I used vorbiscomment.

130 oyla@phineas ~/Music/test % vorbiscomment test_image.ogg
TITLE=Schneller
ARTIST=Die Vorboten
ALBUM=Existenz
DISCNUMBER=01
TRACKNUMBER=03
TRACKTOTAL=14
METADATA_BLOCK_PICTURE=AAAAAwAAAAppbWFnZS9qcGVn...

However, it feels that it adds a second mjpeg (?) video track with the picture, which is probably a different way of adding picture information. I suspected that this was more likely to cause problems, so I removed it via:

ffmpeg -i test_image.ogg -vn -c:a copy output.ogg.

But it seems I am wrong and this is the same thing and in practise how ffmpeg handles this METADATA_BLOCK_PICTURE Meta field (please correct me). In result the METADATA_BLOCK_PICTURE is also gone, not what I wand:

130 oyla@phineas ~/Music/test % vorbiscomment output.ogg
TITLE=Schneller
ARTIST=Die Vorboten
ALBUM=Existenz
DISCNUMBER=01
TRACKNUMBER=03
TRACKTOTAL=14
encoder=Lavf59.27.100

In my test, however, I also noticed that the VLC player uses some kind of caching for images. When I changed the cover image, the old image was always displayed - even after restarting the player. So I looked for an alternative and discovered the smplayer. It really doesn't win any beauty prizes, but it seems to always show the current image.

So I tried the full chain with the test_image.ogg in the Playlist and checked what is happening in the stream at the end. The problem is that the tooling is not explicit enougt in the result.I get the Stream using wget and save the file. Then I look with different tools

ffprobe shows not the Metadata (as expected like it seems it is not able to?)

 Duration: N/A, start: 99.720000, bitrate: N/A
  Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp
    Metadata:
      album           : Existenz
      artist          : Die Vorboten
      title           : Schneller
      track           : 03

vorbiscomment shows no metadata_block_picture

tracknumber=03
title=Schneller
artist=Die Vorboten
album=Existenz
ENCODER=Liquidsoap/2.3.0+dev (Unix; OCaml 5.2.0)

smplayer is not showing any image, also not directly playing the stream from icecast

2. Theory: There is some filtering in liqudisoap

I searched a bit through the source code of Liquidsoap for keywords and came across a function that seems to filter things.

let conf_recode_excluded =
  Dtools.Conf.list
    ~d:["apic"; "metadata_block_picture"; "coverart"]
    ~p:(conf_recode#plug "exclude")
    "Exclude these metadata from automatic recording."

I suspect “recording” is a spelling mistake for “recoding” . The context of the code is not primary the encoding, it is from the "request" part. But this should be used e.G. by playlist() and/or harbor icecast?

liquidsoap/src/core/request.ml

Line 341 in 4587675

let conf_recode =

Maybe there is something happening? But I'm at my wit's end now and I'm running out of approaches and ideas, it's a bit complex and has many aspects. What actually surprises me is that I can't find an existing example of this anywhere. I can't be the first to simply want to “push” an image through to the client :D I am grateful for every thought!

[gAlleb]

Interesting indeed.

regarding

I searched a bit through the source code of Liquidsoap for keywords and came across a function that seems to filter things.

This has a dedicated setting:

Exclude these metadata from automatic recording.

settings.request.metadata_decoders.recode.exclude := ["apic", "metadata_block_picture", "coverart"]