How to "peek" the frames at a specific graph node during playback?

[AdrianTesla]

Hello! I am impressed by your library, seriously, I am totally blown away.
Since I love audio and graphics I decided to create a music graphics visualizer.
For this, I first needed the raw audio data from ma_sound objects, which I got using

ma_data_source_read_pcm_frames(
ma_sound_get_data_source(&sound), ...);

Is there a simpler way? In any case, all is good, I can manipulate graphics using this raw data and the current ma_sound cursor position.
But I wanted more, so I started doing more experiments with the audio graph, creating a LPF and a HPF filter.
This triggered my desire to obtain in some way the most recent frames that has been sent to the device at each of these nodes in the graph, so I can do graphical stuff that follows, for example, the low frequencies, and others that follow the high frequencies, all while the sound is playing.
I couldn't find a way to do this other than running raw ma_lpf/hpf filters on the raw source data myself, using ma_lpf_process_pcm_frames() or similar.
I also tried

ma_node_read_pcm_frames()

but It doesn't even compile (is it an internal thing?), and I think this is supposed to be called on the audio thread by the engine anyways.
This induced me to create a ma_device ad use it to initialize the ma_engine, so I created my own data_callback. Inside there I called ma_engine_read_pcm_frames() which already gives me access to the latest filtered frames that are about to go to the device, basically I get to peek the endpoint of the graph. Since this is the audio thread, I sent those to the main thread using a queue, and got them available the next application frame.
This is also cool, I managed to make graphics that represents what is currently being played by the device after all the filters! But I would really love to do the same using a specific node as the source, not only the endpoint. Is this possible?
May I ask some advices on how to manage such things? In general I'm having hard time accessing the raw audio data dynamically.
I apologize for my ignorance, hoping my request makes sense.
Thank you!

[mackron]

The ma_data_source_read_pcm_frames(ma_sound_get_data_source(&sound), ...) pattern should only be used if the sound is not currently being played. If you do that while the sound is playing you'll end up in a nightmare of threading issues because the audio thread's going to be doing that at the same time. I don't recommend that approach. If you want to inspect audio data as it's being played you would typically want to plug the output of the ma_sound object into the input of a custom node which is where you would do you analysis part. So the short answer to your question is plug in a custom node and do your processing there. That'll be run from the audio thread so you may want to send that through to another thread before processing like you already suggested. The reverb node in the extras folder might give you some direction on how to do a custom node: https://github.com/mackron/miniaudio/tree/master/extras/nodes/ma_reverb_node

Don't call ma_node_read_pcm_frames() directly. That's an internal function. The node graph is designed so that you read from the endpoint (by reading from ma_node_graph).

In the next big release of miniaudio (version 0.12) you'll be able to plug in a callback to inspect the data. There's no timeframe on this yet.

[AdrianTesla]

Interesting, so which is the preferred way of reading the raw frames from a loaded sound? Not necessarily during playback: maybe I could specify some kind of flag in my code to store a copy of the raw data if I need it at loading time? Then I can use it later for any reason.

For the node reading, if I understood correctly: if I wanna "listen" to what is currently being played at a specific node on the graph, I use that node as a source for the input of my custom "listener" node. This listener node should have an input and not output anything to the speakers? Just send me the data from the audio thread in some way. Is this correct?
It doesn't seem quite right, because in the documentation you mentioned that the reading starts recursively from the endpoint of the graph, pulling data and mixing from the attached nodes until some data source is encountered. With this logic, if my custom node doesn't output any data, how can the data be pulled through?
I apologize if I am misunderstanding how it really works, it's a very complex system for me.
As a nice coincidence, i was just experimenting with ma_reverb_node just yesterday :D
I'll go deeper into how to create a custom node as you suggested, can't wait for the next big update of the library.
Thank you so much for the fast and detailed answers.

[mackron]

Your understanding is correct for the node thing. Imagine the most simple of setups - you have 1 sound and that is, by default, connected to the endpoint:

Sound --> Endpoint

You can plug in a node in the middle to do your processing:

Sound --> Custom Processing Node --> Endpoint

When you implement a custom node, you implement a callback to do your processing. That callback takes a buffer to both the input data and output data. You would take the input data and do something with it, and then simply copy the data over to the output buffer. But even better, when you set up your custom node, you can set the MA_NODE_FLAG_PASSTHROUGH flag in your node's vtable which tells miniaudio the input data should pass straight though to the output buffer. This saves you on the cost of copying from the input buffer to the output buffer.

Another thing you can do is use a splitter:

Sound --> Splitter +-----------------------------> Endpoint
                   |                             ^
                   |                             |
                   +--> Custom Processing Node --+

With this technique, you can output silence from your processing node, or preferably, use the MA_NODE_FLAG_SILENT_OUTPUT flag in your custom node's vtable. The splitter is making a copy of the input signal and sending one to the endpoint, which is what will be played back, and sending another copy to your custom node. The custom node still needs to be connected to the endpoint, but since it's outputting silence, it won't contribute to the final mix.

With the reading audio data from a sound thing, there's a lot of things you can do. If you're not playing back the sound, I don't recommend using a ma_sound object. Instead just use a regular old ma_decoder. A ma_decoder is a data source, so it can be plugged into any function that accepts a ma_data_source*. That's a much more direct way of getting audio data. You won't have the reference counting offered by the resource manager, but if you need that you can just use the resource manager directly. You can even do something like load your sound file's audio data into memory, and then initialize two ma_decoders - one for doing your processing, and another to act as your underlying data source of your ma_sound object for playback which you can do by using ma_sound_init_from_data_source(). Plenty of options and flexibility for managing all this stuff in miniaudio.