Recommended way to Buffer a MediaSource (if at all)

[hasezoey]

I am still new to Symphonia, and i have been wondering what the approach to buffering a Media-Source should be, i have read that MediaSourceStream theoretically buffers somewhat, but on larger files it seems to actually not buffer much.
For example on a file with video + audio + metadata (which is around 62MiB and length about 2:58 (mm:ss)), it calls syscall read 4 to 6 times a second, which does not seem right. (and using a higher MediaSourceStreamOptions(4Mib) does not seems to affect anything)
So i had tried making a MediaSource with BufReader(4Mib) in-between, which reduced the times of syscall read on the same file to once every 5-6 seconds.
Additionally on a smaller file audio + metadata (which is around 3.8Mib and length 2:15 (mm:ss)), it calls syscall read once every 2-3 seconds.
And on the BufReader implementation it only calls syscall read once.

And seeking also seems to always have at least one syscall read (which should be to my understanding supported without syscall read)

System: Linux Manjaro
symphonia 0.5.1

the syscalls have been seen via strace

Some more on the strace outputs

the following has been captured on the 62MiB file

No custom buffering (but using MediaSourceStreamOptions { buffer_len: 1024 * 1024 * 4, })

[pid 160423] read(42, "<byte data>"..., 32768) = 32768

(32768 = 32 KiB)

whereas the bufreader does:

[pid 163347] read(40, "<byte data>"..., 4194304) = 4194304

(4194304 = 4096 KiB (4Mib))

BufReader impl

use std::fs::File;
use std::io::{BufReader, Read, Result, Seek, SeekFrom};

use symphonia::core::io::MediaSource;

/// Buffer capacity in bytes
/// 
/// 1024 * 1024 * 4 = 4 MiB
/// 
/// [`BufReader`] default size is 8 Kib
const BUF_SIZE: usize = 1024 * 1024 * 4;

/// A [`MediaSource`] for [`File`] but using a [`BufReader`] (Buffered file source)
pub struct BufferedSource {
    /// The inner reader
    inner: BufReader<File>,
    /// Cache the [`MediaSource::is_seekable`] call
    is_seekable: bool,
    /// Cache the [`MediaSource::byte_len`] call
    byte_len: Option<u64>,
}

impl BufferedSource {
    pub fn new(file: File) -> Self {
        let mut is_seekable = false;
        let mut byte_len = None;

        if let Ok(metadata) = file.metadata() {
            // If the file's metadata is available, and the file is a regular file (i.e., not a FIFO,
            // etc.), then the MediaSource will be seekable. Otherwise assume it is not. Note that
            // metadata() follows symlinks.
            is_seekable = metadata.is_file();
            byte_len = Some(metadata.len());
        }

        let inner = BufReader::with_capacity(
            BUF_SIZE,
            file,
        );

        BufferedSource {
            inner,
            is_seekable,
            byte_len,
        }
    }
}

impl MediaSource for BufferedSource {
    fn is_seekable(&self) -> bool {
        self.is_seekable
    }

    fn byte_len(&self) -> Option<u64> {
        self.byte_len
    }
}

impl Read for BufferedSource {
    fn read(&mut self, buf: &mut [u8]) -> Result<usize> {
        self.inner.read(buf)
    }
}

impl Seek for BufferedSource {
    fn seek(&mut self, pos: SeekFrom) -> Result<u64> {
        self.inner.seek(pos)
    }
}

i also encountered some hiccups in playback, but i am not sure if this is something wrong with the application or symphonia, but using the custom BufRead in-between seems to have at least lessened those hiccups (i know that it is not a problem with my system, because mpv and other players play fine)

[pdeljanov]

MediaSourceStream is like BufReader in that one of it's goals is to buffer enough data such that it doesn't need to read from the MediaSource too often. Unless the MediaSource is an io::Cursor, this naturally reduces the system call overhead. MediaSourceStream will dynamically adjust its read-ahead size (the size of any one read from the MediaSource will be) between 1 and 32kB. Therefore, if you use a BufReader with a capacity < 32 kB (default is 8 kB), then there's no benefit.

MediaSourceStream will not read more than 32 kB at a time because it would then introduce too much latency for real-time sources (e.g., live streams). For example, if the media bitrate is 128 kbps (16 kB/s), a 32 kB read is 2 seconds of audio which is 2 seconds of latency if the source is producing it in real-time. Naturally, the latency gets worse as the read-ahead size increases. 32 kB is decently balanced.

Internal to MediaSourceStream, the data read from the MediaSource is copied into a ring-buffer. The size of this buffer is what is controlled by MediaSourceStreamOptions. This supports a second goal which is allowing limited seeking for unseekable MediaSources (e.g., standard input, HTTP stream, etc.). Modifying it will not reduce the number of system calls.

---

The likely reason you are having audio stutters is not this buffering.

To ensure glitch-free playback you will need to have a decode thread that decodes audio as quickly as possible and then buffer the decoded audio into a ring buffer. Then, the audio playback thread will read decoded audio from this ring buffer. Typically, this ring buffer will be lock-free as well since the audio playback thread is real-time and should never be stalled for an unbounded period of time.

symphonia-play is a very bad media player in this regard. If you are experiencing problems with it, then this is likely the reason. You may be able to workaround it by increasing your PulseAudio or PipeWire latency.

[hasezoey]

thanks for the response, in my case it is a static file from the music library (not a realtime stream) that i am trying to "optimize" (less hiccups / syscalls); which to my understanding is a fine use-case to use extra buffering before MediaSourceStream

symphonia-play is a very bad media player in this regard. If you are experiencing problems with it, then this is likely the reason. You may be able to workaround it by increasing your PulseAudio or PipeWire latency.

thanks for noting this, though i am not using symphonia-play, i am working somewhat on termusic (or more specifically tramhao/termusic#206), which has a old modified clone of rodio, which i am trying to wrap my head around (that is why i said i dont know if the hiccups are from symphonia or the application)

To ensure glitch-free playback you will need to have a decode thread that decodes audio as quickly as possible and then buffer the decoded audio into a ring buffer. Then, the audio playback thread will read decoded audio from this ring buffer. Typically, this ring buffer will be lock-free as well since the audio playback thread is real-time and should never be stalled for an unbounded period of time.

so if i understand correctly, i need to have:
1 thread for realtime playback (i think this is handled by cpal),
1 thread for decode (and likely reading from the file / source)
1 main thread for commands

if that is true, then i will somehow need to find out how to update the mentioned modified clone of rodio to decode in a separate thread than the playback thread and/or debug more why the hiccups are happening