0.8.0 #122
Open
0.8.0 #122
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Added "is_consumer_alive" flag to ensure graceful termination of the ConsumerThread
* Update dependency to allow for ovos-config~=0.1 * Update upload-artifact action * Update URL used in unit tests * Update default config URL --------- Co-authored-by: Daniel McKnight <daniel@neon.ai>
* async consumer using aio-pika * aio-pika dependency * Fixed some minor issues and polished codebase * Made aio-pika extra dependency and updated setup flows * updated README.md with aio setup instructions * Extracted consumers to the separate files, made Import handling for AsyncConsumer --------- Co-authored-by: NeonKirill <kirill.grim@gmail.com>
…on (#100) * moved `create_connection` as a separate function in `AsyncConsumer` * Improved aio handling * Removed non-working tests and added pika-based Async Consumer * Small improvements to handle failure scenarious * tmp commit * small fix * Updated codebase to fully test SelectConsumerThread * small fix * Removed parameterized from test_connector.py * Fixed tests and restored logic to pre-aio state * small fix * Added license notices --------- Co-authored-by: NeonKirill <kirill.grim@gmail.com>
…ful restarts handling (#102)
* Improved logic to create new connection instance on running consumer thread
* Refactor tests to use a local RMQ instance instead of configured remote Allow non-backwards-compat tests to run in parallel * Fix indent error in `unit_tests` * Fix indent error in `unit_tests` * Update async test to support changes * Remove duplicated backwards-compat test automation * Refactor test classes to match module order for better readability Add test coverage for `utils` submodule Update `create_mq_callback` to support decorating both functions and methods (including static methods) * Update docstring for `create_mq_callback` Fix bug introduced in test refactoring * Add test coverage for `BlockingConsumerThread` * Refactor test names and simplify GHA automation * Add basic unittest coverage for SelectConsumerThread lifecycle Update SelectConsumerThread to better define startup/shutdown processes Add type annotations to SelectConsumerThread Add logging to SelectConsumerThread * Address all review comments Remove redundant `channel.stop_consuming` call Revert change to `super()` call
* Update `SelectConsumerThread` to pass exceptions to `self.error_func` to match `BlockingConsumerThread` Handle channel/connection closed exceptions explicitly in `SelectConsumerThread` * Update `BlockingConsumerThread` to ensure connection is closed exactly once Update tests to check for expected error callbacks * Ignore `StreamLostError`s during consumer shutdown
* Fix SelectConsumer reconnection with unit test coverage * Add more details to TODO comment Add sleep to unit test to troubleshoot failure * Remove unused added Lock * Update _close_connection to ensure connection is closed before joining the thread Remove unused import per review
* Prevent attempting to restart an exclusive consumer (will fail repeatedly) * Catch errors during shutdown to suppress extra errors in Sentry * Log async consumer param with other registration logs * Mark subscriber not queue_exclusive to allow for consumer restarts * Ensure SelectConsumerThread has an available asynctio event loop Troubleshooting https://neon-ai.sentry.io/issues/6036439787/events/2e96541936804a5a94a33c55acd3ef4f/ * Explicitly pass io_loop to `SelectConnection` to troubleshoot https://neon-ai.sentry.io/issues/6036439787/events/e198c87d13c54295adf30a0d12cc0780/ * Update asyncio ioloop handling * Update asyncio ioloop handling (https://neon-ai.sentry.io/issues/6222347480/events/487e1f7cbb284c8a8db2aa96d95e8d9a/) * Troubleshooting IOLoop handling https://neon-ai.sentry.io/issues/6222347480/events/6e3b0576b320403bbf60ab01fdd9c05a/ * Troubleshoot IOLoop init in SelectConsumerThread * Ensure ioloops defined in `run` are started * Update event loop handling to ensure a maximum of one loop per SelectConsumer Ensure any created IOLoops are stopped when the thread is joined
…nable errors (#112) Refactor connection close error log to accurately state "closed" instead of "closing" https://neon-ai.sentry.io/issues/6117742740/events/8741afbd379f439a843d69094c0a63fe/
Add test of MQ restart logic Add `started` property to easily check if an `MQConnector` instance is started
* Update default async behavior to `True` Read async behavior from envvar `MQ_ASYNC_CONSUMERS` Update documentation to reflect changes in default and configuration * Update logging Annotate `NeonMQHandler` class and explicitly disable async consumers * Update documentation code snippet to match description
* Update `emit_mq_message` to support SelectConnections Related to neon-iris improvements * Update `emit_mq_message` to support SelectConnection channel creation callback * Loosen dependency to allow newer `pika` * Troubleshoot channel opening * Add more logging * Add more logging to troubleshoot blocking connection usage * Cleanup logging Troubleshoot blocking connection usage * Update `emit_mq_message` to prevent mutating input data Add test coverage for `emit_mq_message` for Blocking and Select connections * Update log to DEBUG Disable sync and observer threads in test Connector instance * Update test to wait for open event
* Add exception handling for RMQ startup * Add helper to test RMQ server availability with a test connection instead of allowing all handlers to fail when RMQ is not ready at startup * Refactor to always test RMQ connection before asserting MQ is ready * Update RMQ connection check error handling * Simplify RMQ connection check error handling * Fix call to `wait_for_startup` to pass valid credentials * Update RMQ availability check exception handling * Implement RMQ native retry mechanism instead of manual implementation * Update logging to `DEBUG` for easier log readability Override `pika` logging explicitly during connection tests to prevent expected errors from creating Sentry issues * Add some extra time to wait for RMQ startup after a successful connection Refactor initial connection retry to try and catch RMQ startup * Reduce warning log to debug to prevent spam during connection start * Revert extraneous changes Address open TODO * Annotate Consumer `error_func` methods and fix invalid test * Annotate `check_rmq_is_available` and stub unit test
Add test coverage for `check_rmq_is_available` Update logging
…sses that also extend `Thread` (#120)
NeonDaniel
approved these changes
Jan 29, 2025
15 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelog
0.7.2a21 (2025-01-27)
Full Changelog
Merged pull requests:
message_idis not None #121 (NeonDaniel)0.7.2a20 (2025-01-24)
Full Changelog
Merged pull requests:
threading.Thread#120 (NeonDaniel)0.7.2a19 (2025-01-24)
Full Changelog
Merged pull requests:
0.7.2a18 (2025-01-23)
Full Changelog
Merged pull requests:
0.7.2a17 (2025-01-23)
Full Changelog
Merged pull requests:
0.7.2a16 (2025-01-23)
Full Changelog
Merged pull requests:
emit_mq_message#115 (NeonDaniel)0.7.2a15 (2025-01-23)
Full Changelog
Merged pull requests:
0.7.2a14 (2025-01-22)
Full Changelog
Merged pull requests:
0.7.2a13 (2025-01-17)
Full Changelog
Merged pull requests:
0.7.2a12 (2025-01-16)
Full Changelog
Merged pull requests:
0.7.2a11 (2025-01-14)
Full Changelog
Merged pull requests:
shutdownmethod for NeonMQHandler with unit test coverage #110 (NeonDaniel)0.7.2a10 (2025-01-13)
Full Changelog
Merged pull requests:
reconnectionwith unit test coverage #109 (NeonDaniel)0.7.2a9 (2025-01-02)
Full Changelog
Merged pull requests:
0.7.2a8 (2024-12-11)
Full Changelog
Merged pull requests:
0.7.2a7 (2024-12-05)
Full Changelog
Merged pull requests:
0.7.2a6 (2024-12-05)
Full Changelog
Merged pull requests:
0.7.2a5 (2024-12-03)
Full Changelog
Merged pull requests:
0.7.2a4 (2024-11-21)
Full Changelog
Implemented enhancements:
Merged pull requests:
0.7.2a3 (2024-09-12)
Full Changelog
Merged pull requests:
0.7.2a2 (2024-04-25)
Full Changelog
Merged pull requests:
0.7.2a1 (2024-03-30)
Full Changelog
Fixed bugs:
* This Changelog was automatically generated by github_changelog_generator