[Off-topic?] Inserting new data into tables

[crabmusket]

Sorry for what may be a tangential question, but:

Is it possible to use Arquero efficiently in cases where data needs to be updated incrementally? I'm currently working with lots of ad-hoc table-like structures in a CAD application. A lot of our operations look like relational verbs, but we also need insertion and update functionality.

A use-case for insertion/update which is closer to Arquero's focus on pure data analysis might be e.g. realtime charts or tickers.

I'd really appreciate any thoughts or recommendations for alternative technologies.

I think "sqlite in-memory database in the browser" is close to what I want. We could theoretically use SQL.js for that, but introducing the SQL language itself is overkill. The way Arquero provides relational-algebra verbs as a library is perfect for our application. I will of course continue make our "ad hoc" relational code work well, but it'd be nice to introduce a library like Arquero to make the structure more clear and to get all the benefits of a well-designed, documented, and fast library.

I'd be interested also whether Arquero's core abstractions might support me "plugging in" something like a MutableColumnTable class and re-using the existing features like query generation.

(We've considered using something like an ECS library, and may end up going down that route. But ECSs are often designed to have entities constructed from arbitrary and dynamic combinations of components. We really don't need that level of dynamism; a known-ahead-of-time table structure is more than adequate. I also considered building our own relational-data library before finding Arquero and realising that you've already implemented a lot of the things I would have wanted, like code generation for queries and reify.)