How to play live audio data using playback

[jitendras7]

Hi @mackron,
I am new in audio programing and not have a good understanding of audio data.

but as per your example, I saw that we use a decoder for play a file and send the decoder reference to the data callback and play until the data of decoder is not NULL. (its my understanding)

I am working on UDP live audio streaming I get the audio data in frames from UDP socket and I want to play that data using miniaudio. can you suggest me the helpful API with working information of function (What should I use).

I am getting data in frames like 324 frames per 10ms (1296 bytes per 10ms) because my audio frame rate is 32400 frame/s

[mackron]

This should be fairly simple. The device and decoder are fully decoupled. I just use the decoder for the sake of the example so some kind of sound is output. The device will run continuously until you stop it. When there is no data to output you can just not copy anything to the output buffer of the data callback and it'll just output silence.

In your case, set the sample rate of the device (via the device config) to 32400. Also set your channel count and sample format. Then in the data callback simply copy over your data into the output buffer. I suppose you'll want to receive your data into some intermediate buffer and then from the data callback read from that buffer which you then output to the output buffer provided by the data callback. That part is just standard buffer management.

Hope that helps.

[jitendras7]

Thanks @mackron for your valuable explanation. I tried this but I cant get the proper output. I just copy the UDP buffer to the pOutput.
Is any size dependency?

Please look my code and suggest any changes if needed

    #define MINIAUDIO_IMPLEMENTATION
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #include <sys/socket.h>
    #include <sys/types.h>
    #include <netinet/in.h>
    #include <arpa/inet.h>
    #include "miniaudio.h"
    
    unsigned char buff[1296];
    char *ip = "10.0.0.1";
    int port = 5000,n;
    int sockfd;
    struct sockaddr_in server_addr, client_addr;
    socklen_t addr_size;
    
    
    void data_callback(ma_device* pDevice, void* pOutput, const void* pInput, ma_uint32 frameCount)
    {
        memcpy(pOutput,buff,1296);//frameCount*pDevice->playback.channels*ma_get_bytes_per_sample(pDevice->playback.format));
    }
    
    int main(int argc, char** argv)
    {
        ma_device device;
        ma_device_config config = ma_device_config_init(ma_device_type_playback);
        config.playback.format   = ma_format_s16;   // Set to ma_format_unknown to use the device's native format.
        config.playback.channels = 2;               // Set to 0 to use the device's native channel count.
        config.sampleRate        = 32400;           // Set to 0 to use the device's native sample rate.
        config.dataCallback      = data_callback;   // This function will be called when miniaudio needs more data.
        
            if (ma_device_init(NULL, &config, &device) != MA_SUCCESS) {
                printf("device initialzation failed\n");
                return -1;  // Failed to initialize the device.
            }
        
        sockfd = socket(AF_INET, SOCK_DGRAM, 0);
            if (sockfd < 0){
                printf("socket error\n");
                exit(1);
            }
    
        memset(&server_addr, '\0', sizeof(server_addr));
        server_addr.sin_family = AF_INET;
        server_addr.sin_port = htons(port);
        server_addr.sin_addr.s_addr = inet_addr(ip);
    
        n = bind(sockfd, (struct sockaddr*)&server_addr, sizeof(server_addr));
            if (n < 0) {
                printf("bind error");
                exit(1);
            }
        addr_size = sizeof(client_addr);
        ma_device_start(&device);     // The device is sleeping by default so you'll need to start it manually.
    
        while(1)
          {
            n=recvfrom(sockfd, buff,1296, 0, (struct sockaddr*)&client_addr, &addr_size);
            if(n<0)
            {
                printf("recive error");
                continue;
            }
            //memset(buff,0,1000);
          }
            ma_device_uninit(&device);    // This will stop the device so no need to do that manually.
            return 0;
    
    }

[mackron]

So a few things to explain. First or all, and most importantly, your network traffic and the device are completely decoupled so you need to architecture your code around that. As soon as you start the device, the data callback will be getting fired continuously, entirely independent of your networked data. What you've got at the moment looks like some kind of tight coupling between your network data and the data callback. That will not work the way you're expecting. Instead you need to use the concept of a producer and a consumer. The producer is what is writing to the intermediary buffer (the data you get from recvfrom()) and the consumer is what is reading from the intermediary buffer, which would be the data callback. The producer and consumer would each have their own cursor/pointer that advances as you produce or consume. In miniaudio there is a special type of buffer called ma_pcm_rb (the "rb" means ring buffer) which is designed for this kind of scenario. Documentation can be found here: https://miniaud.io/docs/manual/#RingBuffers. You would want to set this something to much bigger than 1296. I would probably set it to at least a second worth of audio data (you can use the sample rate to calculate it - it's frames per second). When it's full, just delay reading from the network until some space is available - it just means you are receiving data over the network faster that you're playing it through the speakers.

Another thing, is your network traffic raw PCM data? As in, it's not a MP3 or anything? If it's not raw PCM data, it won't work. You would need to find a way to decode the data before outputting it to your intermediary buffer.

Another mistake you've made is hard coding "1296" in your data callback. You should always use frameCount to determine how much data to output. I can see your commented out line - that looks like a correct solution.

Finally, just for your information, when the miniaudio data callback is fired, the backend is telling you that it needs more data so it knows what to play out of the speakers. It needs the data straight away - you cannot be spending a lot of time sitting in the data callback. It does not wait for anything, and that includes your network traffic, so don't be tempted to just wait in the data callback for your data to arrive. I would buffer a certain amount of audio data up to an acceptable latency before starting playback so you've got enough of a buffer in case you have a stall in your network traffic or whatnot.

[jitendras7]

Yes my data is raw PCM data and 1 frame has 4 bytes and I got the 324 frames( 1296 bytes in every 10ms) from UDP network

I also use the ring buffer (size of ring buffer is 32400*4=129600) and every 324 frames after writing using the ma_pcm_rb_acquire_write() in ring and commit using ma_pcm_rb_commit_write().

and inside the data call back I use the same for read but I got same output like previously

can you please share some example for ring buffer

[mackron]

I don't have an example on hand, but yes you want to do acquire_write(), copy into the pointer it gives you, then commit it with commit_write(). You then then do the same kind of thing on the reading side from the data callback with acquire_read() and commit_read().

What kind of sound are you hearing? Is it total silence? Do you hear any semblance of the original sound at all, or is it totally imperceptible?

[jitendras7]

Yes I hearing some sound but it is not good may be missing some data or may be problem of sync the data, I write a code in python for reading the socket data and play the sound using pyaudio library so its look clear but as you know I want to do this in c code but when I use miniaudio the sound the totally deferent.

in pyaudio i use queue to put reading the chunks since my data coming in chunks (324 frames in one time)

[mackron]

It's a good sign that you can at hear some kind of sound. The queue thing you mentioned from pyaudio is the equivalent of the intermediary buffer that I mentioned in my previous response. The problem is miniaudio doesn't have a data structure that's identical to the queue you're using in pyaudio so it's not a one-to-one translation from Python to C. You'll need to come up with a way to manage the buffering part yourself.

The ring buffer I mentioned earlier is just one way to do it. You could also literally just push your data into a dynamically expanding array if you wanted (just make sure you use a mutex to keep it thread safe). Or you could play around with the ma_data_source chaining system (would also require a mutex for thread safety). There's many ways you could do it.