[Community] [Audio Context] Procedural audio generation

[aliustaoglu]

This extension uses Audio Context API to bring procedural audio generation to GDevelop. This is a JavaScript-heavy extension.

So far you can create:

• Oscillator of type sine, triangle, square & sawtooth
• Assign an ADSR envelope to an oscillator
• Add detune values to an oscillator so it sounds like a synth
• All 12 notes on 8 octaves are defined in the extension so you don't need to calculate frequencies manually
• You can create chords using these notes
• You can play separate notes or chords on these synths/oscillators
• You can add filters (lowpass, highpass, etc.) to synths. By tweaking these values you can get percussion kinda sounds from synths.

Basic help and an example with a full source are available here (I'll probably add more info to this page):

https://aliustaoglu.itch.io/audiocontext-gdevelop-extension
https://liluo.io/instant-builds/45d1f15c-1043-495a-b86a-0b90d7fac49b

[D8H]

Very cool extension!
If you want, you can also submit the example to the example repository. It will be useful for extension users to understand how to use this extension. It also helps to speed-up the review because it generates a link to open it with the web-app directly.

I have some suggessions:

• SynthName parameters could be an Identifier instead of a String. It allows to give autocompletion.
• The code could be a class. Classes can be defined like this:

const MyClass = /** @class */ (function () {

  function MyClass() {
    /**
     * @type {number[]}
     */
    this.myAttribute = [];
  };

  /**
   * @param {number} myParameter A parameter.
   */
  MyClass.prototype.myMethod = function (myParameter) {
  };
};

gdjs.__myExtension = gdjs.__myExtension || {};
gdjs.__myExtension.MyClass = MyClass;

Otherwise, there are some naming conventions for extensions:

• Event-function names and parameter names should start with an upper case character.
• Descriptions should end with a dot.

[aliustaoglu]

Hey, thanks. Yeah, I'll submit the example and make the other changes.

[tristanbob]

I've been playing around with this VERY COOL extension!

• How can I control volume?
• How can I stop a note (or all sounds)?
• It seems that the synth duration values (attack+decay+sustain+release) override the duration that a note is played. Is that intentional?
• Can you get sound to work on an iPhone? (it's not working for me)

Thank you again, and great work. :)

[tristanbob]

Have you considered using a framework to provide more features? (GDevelop extensions can use NPM modules)

Tone.js = https://tonejs.github.io/

Pizzicato = https://alemangui.github.io/pizzicato/

WAD = https://github.com/rserota/wad

[aliustaoglu]

Hi Tristan @tristanbob,
Great questions. Let me try to answer:

• How can I control volume?

  • There's something called "gain node" to which the context should be attached. Well, I didn't. Looks like I should have. How can you have an audio controller without controlling the gain right? Noted. I should add an action for users to add a gain node and allow them to change their parameters.

• How can I stop a note (or all sounds)?

  • I had a stop function however it was a bit buggy so I disabled it. I am planning to add this.

• It seems that the synth duration values (attack+decay+sustain+release) override the duration that a note is played. Is that intentional?

  • Wow. Extra great question. I don't have a good answer though. At first, I wanted to use ADSR params as unitless and proportional to the duration. For example for ADSR=1,2,5,2; the attack would be 1/(1+2+5+2) * duration. But then I messed it up. This is also connected to stop sound functionality. I should fix them all altogether.

• Can you get sound to work on an iPhone? (it's not working for me)

  • Can't believe I didn't even test this. I just checked. Doesn't work on mine either. When I check online it says the action should come from a user action first but this is the case for Android and desktop browsers as well. And there are lots of reports people cannot get Audio Context to work on iPhones.

By the way, the builtin "Play sound/music on channel x" doesn't work for iPhones either. Tested on a Cordova built. Have to use "Play sound/music" without the channel option. Both issues might be related but not 100% sure.

• Have you considered using a framework to provide more features? (GDevelop extensions can use NPM modules)
  • I have indeed :) I am playing with tone.js for a while. It's a great library built on top of Audio Context. I'm working on a tool similar to 1bitdragon using GD but it's half extension and half JavaScript. A bit hard to convert it to no-code-friendly functions. Audio Context is just vanilla and easy to visualise its node system in your head and convert to an extension. If I fully get the picture of tone.js then yeah I might think about turning it into an extension. It's too flexible which is good but making it harder to turn it into an extension. I use JS for quite long but with GD not so much. But lately feel like getting the hang of it.

[D8H]

Are you working on a new version?
I think the extension can already be useful as it is. We can merge the current version if you want.

[aliustaoglu]

@D8H Hey. Sorry, a lot going on at the moment so I cannot say I have time for refactoring this. Also, I was looking at the Tone.js library as Tristan suggested, got carried away and already working on something with that but it will take some time until I can turn it into an extension. That library is great and built on top of audio context.

But if you say this is mergeable as is please go ahead and I'll create the tone.js extension with the suggestions and the formatting you mentioned here. Thanks.