Async queries

[Wouittone]

Hello,

I'm wondering if there is any async capability exposed with Ibis?
My particular use case would be for client-side queries that are potentially modified and run dynamically, changing every couple of seconds possibly. For this, I would like to set up an async query, idealy with Ibis, that would then be able to cancel any previously running query as the result would be stale anyways.

I only found some mention of async in deprecation notes, so I guess this isn't possible but I preffered asking anyways, I figure I might not be the only one in this situation :-)

Thanks for the help!

[jcrist]

Ibis currently doesn't expose any async-specific functionality, and my current feeling is that any async-specific functionality would be out of scope for this project.

If you wanted to use ibis with asyncio (or another async framework), you could wrap the ibis execution in a thread using something like asyncio.run_in_executor (https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor). This would avoid blocking the event loop, but wouldn't play well with cancellation.

Alternatively, you could use ibis only for expression -> sql operations, then make use of some other async DB library to handle the actual execution (e.g. https://github.com/MagicStack/asyncpg for postgres).

In [1]: import ibis

In [2]: t = ibis.table({"x": "int", "y": "float"})

In [3]: from ibis import _

In [4]: expr = t.group_by("x").agg(y_sum=_.y.sum(), y_mean=_.y.mean()) # write an expression

In [5]: sql = ibis.to_sql(expr, dialect="postgres")  # compile it to sql

In [6]: print(sql)
SELECT
  t0.x,
  SUM(t0.y) AS y_sum,
  AVG(t0.y) AS y_mean
FROM unbound_table_0 AS t0
GROUP BY
  1

In [7]: # execute the SQL string asynchronously using e.g. asyncpg