Closed
Description
The documentation specifies that Advanced Queueing (AQ) is only supported in thick mode. The request is to add support to thin mode to be able to use asynchronous operations within the same application, which are not possible in thick mode.
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
anthony-tuininga commentedon Dec 21, 2024
Work has been in progress on this front for the past number of weeks. I can't say which version will contain this support yet, but once it is available for testing I will let you know!
cjbj commentedon Dec 21, 2024
@mxdev88 what specific AQ functionality are you using? E.g. array enq/deq, recipient lists, what kind of payload(s) (RAW, JSON), etc?
mxdev88 commentedon Dec 30, 2024
Initially on the consumer side and using deqone / deqmany with RAW payload.
mkmoisen commentedon Jan 7, 2025
Hi @anthony-tuininga, @cjbj
I'm also excited to have AQ in thin mode, thank you for working on it.
So far I am using object payloads and RAW payloads. I would love to use JSON payloads but don't have the latest Oracle version that supports this.
I use single consumer queues, single enqueues and single dequeues. No agents, No recipient lists. I think in the future array enqueue and dequeue would be important.
I also set enqueue options and dequeue options.
mkmoisen commentedon Jan 7, 2025
@anthony-tuininga @cjbj
Would you please explain what the cost is by using AQ in thin mode in the following manner, compared to using thick mode's native AQ support? My guess is that this work around requires twice as much network IO compared to the thick mode's native AQ support.
My guess is that this results in twice as much network IO as compared to using native AQ in thick mode.
This thin-mode workaround will need to invoke an IO from the application server to Oracle, and then when Oracle has a new event, it will send it back over the same IO channel, for a total of two IO's per message.
With Native AQ in thick mode, I would guess that a socket is opened when the queue is created. Then the application server can go to sleep listening to the local socket. When Oracle gets a new message, it pipes it over the network to this socket, at which point the application server wakes up and reads it. So a total of one IO per message.
Is that right?
anthony-tuininga commentedon Jan 7, 2025
I am sure there is some overhead in calling PL/SQL as opposed to using AQ natively, but I don't think it is quite as bad as you think. When you call
dequeue()
, if there is a message available it will be copied immediately into the output variable. If there is no message available and the mode is to wait for one to be available, then the PL/SQL block will "hang" until such time as the message is available and then copy it into the output variable. A similar thing happens when using AQ natively -- you simply avoid the overhead of using PL/SQL. In both native and PL/SQL the connection is not usable until such time as a message is available. I don't know exactly what happens internally, but that much is certain at least!If you want to know for sure what the overhead is, you can try with thick mode -- both natively and with PL/SQL and see what the difference in performance is.
cjbj commentedon Feb 14, 2025
@mkmoisen I don't want to get you too excited but it seems our next release (version 3.0) will have our (initial) Thin mode AQ support, since we just merged into our internal repo. @shivani-ka and her team have been working hard on this, and will continue adding functionality in future versions too. The code is not on GitHub as I type, but could be soon. The release notes say: "Added Oracle Advanced Queuing support for single enqueue and dequeue of RAW and Oracle object payload types." The fine print is that this initial Thin mode support is for classic queues (not transactional event queues). Also JSON and JMS payloads, array message queuing and dequeuing operations, and Recipient Lists are only supported in python-oracledb Thick mode in this version. As to release timing, we have a few non-AQ features close to being ready to land, and a lot of testing to do.
mkmoisen commentedon Feb 14, 2025
@cjbj very glad to hear it. My application may be able to use it, since it uses classic queues, single dequeues, with oracle object payloads.
Thank you guys!
mkmoisen commentedon Feb 14, 2025
@cjbj Do you know in which Oracle version "transactional event queues" was released?
The only documentation I can find for it is Oracle 21:
https://docs.oracle.com/en/database/oracle/oracle-database/21/adque/index.html
The Oracle 19 documentation does not refer to "transactional event queues":
https://docs.oracle.com/en/database/oracle/oracle-database/19/adque/index.html
However this blog post claims that transactional event queues were released in 19:
https://www.oracle.com/database/advanced-queuing/
Add AQ support in thin mode for single enqueue and dequeue of RAW and
anthony-tuininga commentedon Feb 14, 2025
I have pushed a patch that partially implements this enhancement (note restrictions above) and have initated a build from which you can download pre-built development wheels once it completes. You can also build from source if you prefer. If you can test your scenario and provide feedback before the release, that would be appreciated!
mkmoisen commentedon Feb 15, 2025
Hi @anthony-tuininga
How may I download a pre-built dev wheel?
For example under
[Build wheels for windows-latest Python-3.13-x86](https://github.com/oracle/python-oracledb/actions/runs/13333204363/job/37242192476#logs)
I clicked this link:Artifact download URL: https://github.com/oracle/python-oracledb/actions/runs/13333204363/artifacts/2593722184
However I'm getting a 404 on github, not sure if it is public or not.
Thanks.
anthony-tuininga commentedon Feb 15, 2025
Go here: https://github.com/oracle/python-oracledb/actions/runs/13333204363 and then click the "python-oracledb-wheels" link at the bottom under artifacts. This link: https://github.com/oracle/python-oracledb/actions/runs/13333204363/artifacts/2595043763
It is possible that you tried while the builds were ongoing. The separate wheels are built and then combined into a single artifact at the end!
mkmoisen commentedon Feb 17, 2025
Hi @anthony-tuininga @cjbj
I have installed it on local but I am receiving this error when calling
queue.deqone()
:I'm using Oracle Database 19.
I'll create a script to minimally reproduce this and report back.
mkmoisen commentedon Feb 17, 2025
@anthony-tuininga @cjbj
I'm using:
Here is how I made my queue in Oracle:
Here is how I am calling in Python:
Here is the error I'm getting while calling
queue.deqone()
:It works fine in thick mode.
Thank you.
shivani-ka commentedon Feb 18, 2025
@mkmoisen I am looking into this and will update once I have findings.
cjbj commentedon Feb 18, 2025
@mkmoisen this was patched in a092cfc. Thanks for the report.
mkmoisen commentedon Feb 19, 2025
@cjbj @anthony-tuininga @shivani-ka
I've tried the new patch and it appears to be working smoothly on my local.
Thanks!
anthony-tuininga commentedon Mar 3, 2025
This has been included in python-oracledb 3.0.0 which was just released.