Access Koto AST via Rust API

[cornedriesprong]

Hi! First, thanks for creating Koto, what a beautifully expressive language. This is not so much an issue as a question/request for advice, hope that's ok. I'm attempting to use (a subset of) Koto as a DSL of sorts to construct audio processing graphs in my Rust DSP library, in the spirit of Faust or Genexpr. The way I ended up doing this is like so:

Expr = |type, input, args|
  type: type
  input: input
  args: args

sine = |hz|
  Expr "sine", hz, []

mul = |val, input|
  Expr "*", input, [val]

add = |val, input|
  Expr "+", input, [val]

this lets me write, for example:

sine 0.2 >> mul 200 >> add 100 >> sine

In my rust code, this shows up as a nested map

let src = fs::read_to_string("./script.koto").unwrap();
let mut koto = Koto::default();
let graph = match koto.compile_and_run(&src)? {
    KValue::Map(m) => parse_graph(&m),
    other => bail!("Expected a Map, found '{}': ({})", other.type_as_string(), koto.value_to_string(other)?),
}

...which I then parse into a recursive data structure (parse_graph) that my DSP library can construct an audio graph out of.

Now this feels a bit cumbersome and leads me to wonder if there is a way that I can inspect Koto's AST directly from my Rust code? I suppose this would let me use Koto's built in arithmetic operators, for example, instead of defining my own mul and add functions. I don't see such functionality exposed through the Rust API, so I suppose it's currently not possible, but I'm wondering if it's a use case you've considered, of if maybe there is another way to go about this that I haven't realized.

[irh]

Hi! First, thanks for creating Koto, what a beautifully expressive language.

Hi @cornedriesprong 👋 Glad you like it!

This is not so much an issue as a question/request for advice, hope that's ok.

All good! I converted the issue into a discussion so this can be a bit more open ended.

I'm attempting to use (a subset of) Koto as a DSL of sorts to construct audio processing graphs in my Rust DSP library, in the spirit of Faust or Genexpr.

This is really great to see! I've had in mind to try something like this one day (I was thinking about a scriptable front-end for fundsp) but never had the time, I'm happy that you're giving it a go. Please let me know how you get on.

Some thoughts in no particular order:

• Why a subset of Koto? I would have thought that there's a lot of potential in having the full language available?
• If you'd like to go with a subset all the same then you can access the AST by using koto_bytecode::Parser directly, the parse function takes a string and returns the AST. Then you make your own compiler for processing graphs by traversing the AST.
• If it was me I think I would first try creating some kind of 'audio node' object in Rust, which would represent your graph connections, and then make it available in Koto scripts by implementing KotoObject. That way you wouldn't have to decode the map structure yourself, you'd already have it built up in your rust objects after the script is run.

[cornedriesprong]

Thanks for your reply @irh!

Why a subset of Koto? I would have thought that there's a lot of potential in having the full language available?

I figured I wouldn't need the full language to express the kinds of manipulations I want to express, but then again it wouldn't hurt to have it available either. I guess I'm still torn between using Koto's parser as the basis for my own DSL or embedding the entire language runtime in my audio engine.

If it was me I think I would first try creating some kind of 'audio node' object in Rust, which would represent your graph connections, and then make it available in Koto scripts by implementing KotoObject. That way you wouldn't have to decode the map structure yourself, you'd already have it built up in your rust objects after the script is run.

This was my initial approach. I guess I'm struggling to see how I can represent an audio graph in Koto as a recursive structure, whilst keeping the syntax as lightweight as in my example above. For instance, the sine generator takes a frequency argument, which can either be a constant value (like 440 Hz) or another generator (for example an LFO). To represent this as a KotoObject, I would need some kind of sum type argument which doesn't seem to exist. In my Rust code it looks like:

enum Expr {
    Operator {
        kind: OperatorType,
        input: Box<Expr>,
        args: Vec<Expr>,
    },
    Number(f32),
}

Another thought I had: the pipe operator is great for chaining audio processors, but I could see myself wanting to split, merge and parallel process signals as well. Seeing that Koto doesn't support operator overloading I'm guessing I'd need to extend the parser to add custom operators for these.

This is really great to see! I've had in mind to try something like this one day (I was thinking about a scriptable front-end for fundsp) but never had the time, I'm happy that you're giving it a go. Please let me know how you get on.

Yeah, fundsp is great! I'm implementing this DSP library mostly as a learning exercise for myself, so I'm just reinventing the wheel over here 🙂. If it goes anywhere I'll make sure to give an update.

[irh]

I guess I'm still torn between using Koto's parser as the basis for my own DSL or embedding the entire language runtime in my audio engine.

If it helps I think of Koto scripts as being like a UI for your app, so the runtime would likely run in your main thread, and then send messages to the audio engine on updates.

To represent this as a KotoObject, I would need some kind of sum type argument which doesn't seem to exist. In my Rust code it looks like:

Function call arguments coming from Koto are received in Rust as KValues, and KValue::Object is the variant that holds Rust types that implement KotoObject.

You could have something like a Node struct that implements KotoObject:

struct Node {
    expr: Expr,
    outputs: Vec<Arc<Node>>,
}

and then you can override KotoObject::call, something like:

impl KotoObject for Node {
    fn call(&mut self, ctx: &mut CallContext) -> Result<KValue> {
        match ctx.args {
            [KValue::Object(other_object)] if other.is_a::<Node>() => {
                // Cast the object to get access to the other `Node`
                let other_node = other_object.cast::<Node>?.unwrap();
                self.connect(other);
                // Return the other node to continue the call chain (or something)
                Ok(KValue::Object(other_object))
            }
            unexpected => return unexpected_args("|Node|", unexpected),
        }
    }
}

Seeing that Koto doesn't support operator overloading I'm guessing I'd need to extend the parser to add custom operators for these.

You can't define new operators but you can overload all existing operators, in Koto using metamaps, and in Rust via KotoObject, (KotoObject::add, ::subtract, etc.).

Good luck! Happy to answer any questions as they come up.

[cornedriesprong]

I ended up having my Expr struct implement KotoObject. That way I can define a Koto function to create a node that can take either a number or another object as its argument:

koto.prelude().add_fn("sine", |ctx| match ctx.args() {
    [KValue::Number(hz)] => Ok(KValue::Object(
        Expr::Operator {
            kind: OperatorType::Sine,
            input: Box::new(Expr::Number(hz.into())),
            args: vec![],
        }
        .into(),
    )),
    [KValue::Object(obj)] if obj.is_a::<Expr>() => Ok(KValue::Object(
        Expr::Operator {
            kind: OperatorType::Sine,
            input: Box::new(obj.cast::<Expr>()?.to_owned()),
            args: vec![],
        }
        .into(),
    )),
    unexpected => type_error_with_slice("a number", unexpected),
});

I can also define arithmetic operations on nodes like so:

impl KotoObject for Expr {
    fn multiply(&self, rhs: &KValue) -> koto::Result<KValue> {
        match (self, rhs) {
            (Self::Number(value), KValue::Number(num)) => {
                Ok(KValue::Number((value * f32::from(num)).into()))
            }
            (Self::Operator { .. }, KValue::Number(num)) => Ok(KValue::Object(
                Expr::Operator {
                    kind: OperatorType::Gain,
                    input: Box::new(self.clone()),
                    args: vec![Expr::Number(num.into())],
                }
                .into(),
            )),
            _ => panic!("invalid multiply operation"),
        }
    }
}

that way I can can compose them like so (modulating the frequency of a sine oscillator with the output of another 3 hz sine multiplied by 100):

(sine 3) * 100 >> sine

I'll keep working on this, if you have any suggestions based on my approach I'd love to hear them of course @irh . Thanks a lot for your help so far.

[irh]

This is great, glad that it's working out for you 👍

If you get to the point where you think it's shareable I'll be excited to have a play with it.

Please let me know if you run into any problems, or have suggestions for improvements, particularly to the Rust API and its documentation.