Feature: Consider swapping Goja with Sobek

[nickchomey]

Feature description

The grafana/k6 team forked Goja because, among other reasons, their PR to add ES Modules support to Goja sat in limbo for far too long.

dop251/goja#430

They clearly have much more resources and motivation than Goja's dev, so it seems that Sobek is the way forward for JS in Go. K6 is already using it in their quite mature and demanding Production environment. It would be great if Conduit could leverage it!

[lovromazgon]

I don't think we will switch just now, mainly for 3 reasons - I think it's too early to tell that the fork will be better maintained than the original repo, it does not solve any particular pain point that we are experiencing, and we already invested time into a more performant approach using standalone processors based on WASM for power users.

I'll keep this issue open though and we can keep an eye on how things develop.

[nickchomey]

That's fine. Though

1. I do think it'll be considerably more actively maintained than goja. They're also merging in anything added upstream in goja, so it's more of a superset than fork.

2. it solves the significant pain point of not being able to use ES modules, which is the standard now. Without those, it's much more difficult to use this mechanism at all.

3. It's not about performance, but accessibility/ergonomics - not everyone wants to (or even knows how to) use wasm. It has become very common to use JavaScript for scripting in other applications - k6 as mentioned, mysql even allows for it now for stored procedures, surrealdb, NATS NEX, and of course any of the 9500 repos that use Goja.

4. There's surely processing use cases that are most (or even only) appropriate with JavaScript - eg things that are related to the web, and can benefit from js libraries.

Anyway, hopefully you'll consider it at some point!

[nickchomey]

Here's their progress on adding up to ES2024 compatibility

However, it might not actually be possible to implement this in Conduit. Sobek's readme says

Also of note is that due to the nature of ESM you need to have an event loop implementation to use it. That is still not provided by Sobek and likely never will be.

Its probably also why it required a larger effort to add ESM to k6, rather than just swapping goja with sobek.

[nickchomey]

I did just find this from goja for providing nodejs compatibility - not sure to what degree it could be used in conduit. https://github.com/dop251/goja_nodejs