FFT audio processing as an effect/filter

[crosswick]

I just realized that currently FFT is only implemented as Audio Sink... I was hoping to use it for audio processing, so with calculation back to audio using iFFT.

I can imagine this being quite a challenge to implement from the current state of the library...

or perhaps there is a workaround of sorts?

[crosswick]

Ah I see Maximilian does AudioStream sink & source processing and includes FFT...

[crosswick]

Actually I only thought it could act as sink because I saw 'signal processing' mentioned... unless I'm overlooking something, the current implementation of Maximilian does not allow input into it.

[crosswick]

Maximilian itself does not handle audio input, I've learned.

[pschatzmann]

I tried to implement some Pitch Shifting using FFT, but it turned out to be slow to be useful.
If you check the AudioRealFFT class, there is a ifft method.

But feel free to extend the functionality ...

[crosswick]

I read that yes - I happen to know that the FFT pitch shifting as documented by Stephan Bernsee (now Stephan Sprenger) is a very perfectionist algorithm; there exist ones that are way more optimised, mainly because they don't try to calculate exact frequency values first.

If I understand correctly, AudioRealFFT currently cannot be instantiated with an output of choice, right? like FaustStream can, for instance. I am not a highly experienced programmer but I might be able to do some useful work in putting together some functionalities.

For now I'm intrigued by Maximilian and will see if that is useful for prototyping.

[crosswick]

@pschatzmann if possible, could you please give a brief overview of what it would entail to add audio output to AudioRealFFT? Is there a comparable class I could reference/copy from?

I'm guessing something like:

• constructor that allows for specifying audio output
• int available() function
• size_t readBytes(const uint8_t *buffer, size_t size) function

[pschatzmann]

Just check the available documentation

[pschatzmann]

In the Basic Introduction there is the explanation of Source, Target and copy...

[crosswick]

I am aware of this - in the case of a 'plugin' object though (passthrough, input & output) I do not understand how I am supposed to define the StreamCopy object if that 'plugin' is not the final copy destination, as it is in the example above.

In the basic introduction the example that comes closest to this is about the VolumeStream class:
https://github.com/pschatzmann/arduino-audio-tools/wiki/Introduction#changing-the-signal

That class does allow to specify a specific output using its constructor; the CustomStream does not. (I realize I was erroneously talking about wanting to specify an input).

So please :) again: if the CustomStream class from the example it meant as a simple passthrough, how do I connect it between two objects...

[pschatzmann]

Make a subclass of a ModifyingStream and allow the definition of the final output in the constructor

Just to get a better understanding: so you want to write to the FFT class and when there is a result you want to do a reverse FFT and output this to the final output ?

[crosswick]

Thank you.

I hope you understand I'm trying my best to work with what's presented - the current explanation of the CustomStream I think honestly is lacking in clarity in this regard - and again, there is a mismatch with the out.begin(info); call. I would appreciate some more profound discussion about things like these other than 'read the Wiki!' - with all due respect.

And yes that is what I want to do, please see my initial comment in this discussion. There is a lot of interesting processing to be done in the spectral realm, of course.

[pschatzmann]

What do you mean with mismatch with with the out.begin(info)?
The FFT is expecting a AudioFFTConfig object.

Try to keep things simple and maybe just start to add a new constructor with a Print parameter e.g. to AudioRealFFT
Starting from there you should be able to figure out what needs to be extended.

• e.g. you will need to add a Print class variable to AudioFFTBase
• this leads that you need a setOutut(Print method) that you can call from the constructor of AudioRealFFT which sets this variable
• in processSamples you need to call the new method rfft(); after fft();
• etc

[crosswick]

in https://github.com/pschatzmann/arduino-audio-tools/wiki/Writing-your-Custom-Input&Output-Class

the begin() method is defined as:

  /// Starts the processing
  bool begin() {
    TRACEI();
    // add your start logic: e.g. start callbacks
    return true;
  }

later in the accopanying sketch, the begin() method is called with an argument: out.begin(info);

Which leads to an error, because the begin() method is not defined to accept an argument.

[pschatzmann]

Usually the following applies:

• begin() is supposed to start the processing just using some default parameters.
• setAudioInfo() is updating just the sample rate, channels, and bits per sample and can be potentially called a any time
• begin(AudioFFTConfig info) starts the processing with some explicit parameters

[crosswick]

Thank you - I will resume my quest later tonight!

[pschatzmann]

I was overtiring the begin() method in AudioFFTBase to follow my description above better

[pschatzmann]

I committed some changes and implemented the ReverseFFT for the AudioRealFFT.
If you want to use it for another implementation, just extend it accordingly...

[crosswick]

Great, I'll have a look, thank you.

[crosswick]

I am looking to do something like this spectral gate as a first test; this exact code does not compile because fft.setBin() does not exist; is there already a typical alternative? I could not find one yet.

#include "AudioTools.h"
#include "AudioLibs/AudioBoardStream.h"
#include "AudioLibs/AudioRealFFT.h"

AudioBoardStream kit(AudioKitEs8388V1); // Access I2S as stream
AudioRealFFT fft(kit);
StreamCopy copier(fft, kit); // copy kit to kit

void fftProcess(AudioFFTBase &fft){
    // Simple spectral gate
    const int numBins = fft.size() / 2;
    const float threshold = 0.1; // Adjust this value to set the gate threshold

    for (int i = 0; i < numBins; i++) {
        float magnitude = fft.magnitude(i);
        
        // Apply gate: if magnitude is below threshold, set it to zero
        if (magnitude < threshold) {
            fft.setBin(i, 0, 0); // Set both real and imaginary parts to zero
        }
    }

    // Optional: Print some information about the gating process
    Serial.println("Spectral gate applied");
    Serial.print("Number of bins: ");
    Serial.println(numBins);
    Serial.print("Threshold: ");
    Serial.println(threshold);
}

// Arduino Setup
void setup(void) {
    Serial.begin(115200);
    AudioLogger::instance().begin(Serial, AudioLogger::Warning);
    
    auto cfg = kit.defaultConfig(RXTX_MODE);
    cfg.sd_active = false;
    cfg.input_device = ADC_INPUT_LINE2;
    kit.begin(cfg);

    // Setup FFT
    auto tcfg = fft.defaultConfig();
    tcfg.copyFrom(cfg);
    tcfg.length = 1024;
    tcfg.callback = &fftProcess;
    fft.begin(tcfg);
}

// Arduino loop - copy data
void loop() {
    copier.copy();
}

[crosswick]

again please, which driver? this one? https://pschatzmann.github.io/arduino-audio-tools/classaudio__tools_1_1_f_f_t_driver.html

no data to work with directly there

[pschatzmann]

Yes, the one that you can request via the driver() method given in my link above

[crosswick]

I don't follow. That driver() method returns a pointer to an FFTDriver object, which in turn does not offer a direct means to work with typical FFT data like real and imaginary values. Does it?

[pschatzmann]

You would need to extend the functioinality of the driver to provide a setBin() and getBin() method
or cast to the implementation and access the driver specific data directly

e.g. in FFTDriverRealFFT you can find

    float *p_x = nullptr; // real
    float *p_f = nullptr; // complex

[crosswick]

You see, when you tell me something like this:

No, you would need to add this.
There is currently only the setValue() to set the real part

Of cause you could alternatively directly work with the data provided in the driver

I think a logical interpretation is for me to think that if I work directly with that data, that I don't need to add any methods. Now you say that I do, so is it really a relatively direct alternative?

Also: how should I have known that those are the real and complex values? Those comments are not in the documentation.

All in all I don't think this library and our interactions about it are working for me, for now. I will look elsewhere to experiment with FFT data on an embedded system. Best of luck to you and thank you.