Binaural audio support

[mklimko-perpetio]

Hi Marco Bavagnoli,

I'm the person who connected with you on LinkedIn. I have an exciting idea that I'd like to discuss with you regarding your plugin.

I've noticed that there aren't many alternatives in Flutter for binaural audio (3D audio) support. While there is one plugin called "surround_sound" (Link: https://pub.dev/packages/surround_sound) that can do something similar, it relies on a webview with JavaScript, limited to moving the oscillator sound with only 3 axes and adjusting the frequency.

My proposal is to add a native 3D sound feature to your plugin. By incorporating binaural audio, your plugin could enable developers to create various applications such as meditation apps, new music players with immersive 3D effects where the sound revolves around the listener's head, and even integrate it with the Flame Flutter package to build games with realistic audio experiences.

Currently, I'm working on a Flutter app that utilizes binaural audio, but it relies on Unity to achieve this effect. However, the Unity integration for Flutter is not stable, and the Unity package itself is relatively large. Implementing the entire Unity package for just one feature, 3D sound, might not be the most optimal solution for every app that requires this functionality. So, I'd love to contribute to your plugin by helping implement this feature. However, I've encountered some challenges while attempting to map sound bytes, and I'm seeking guidance on the correct approach.

I believe adding 3D sound support to your plugin could be a game-changer for the Flutter community, providing developers with a powerful tool to enhance the audio experience in their apps. I'm eager to discuss the feasibility of this idea and collaborate on making it a reality. Let me know your thoughts and how we can move forward with this exciting project.

Looking forward to your response.

[alnitak]

Hi,

my intent for this project, is to have as much control over audio data as possible. I found SoLoud library the right choice to do this. It is born for gaming and it has a lot of potential!
Some days ago I've added microphone support for just reading audio samples, but I was planning to add 3D audio support also! SoLoud provides it, so it should be straightforward add that into the plugin. Look at here https://solhsa.com/soloud/concepts3d.html and here https://solhsa.com/soloud/soloud_20200207.html#d-audio

I need to study it and the way to add, but it is feasible!
I was undecided whether the next move would be to add audio effects or 3D. I think the weekends I'll sketch something for 3D support. I'll make a new branch in the repo. In the meantime we can read about how SoLoud handles 3D audio :)

[alnitak]

Hi,

I have refactored the AudioIsolate class. AudioIsolate class is now named SoLoud and it is in soloud.dart. The isolate spawned method is still in audio_isolate.dart (it's more readable now).

Added almost all 3d audio SoLoud's methods. Available in SoLoud.dart
Added a new tab for testing 3d audio. There is a circle where the listener is placed in the center and a moving siren audio is represented by a little circle which is automatically animated or can be moved by mouse gesture. The sound fade off at the circonference (set with the SoLoud().set3dSourceMinMaxDistance() method). There is also a doppler effect that can be turned off.

With this example, the sound parameters are tested, but I didn't made test with listener parameters which is still and positioned at (0, 0, 0).

I have updated the repo and the news are in the 3Daudio branch! I need to test on win, ios and mac yet.
Let me know!