Releases: aio-libs/aiokafka
Releases · aio-libs/aiokafka
Speed enhancements and Consumer rewrite
Major changes:
- Full refactor of the internals of AIOKafkaConsumer. Needed to avoid several
race conditions in code (PR #286, fixes #258, #264 and #261) - Rewrote Records parsing protocol to allow implementation of newer protocol
versions later - Added C extension for Records parsing protocol, boosting the speed of
produce/consume routines significantly - Added an experimental batch producer API for unique cases, where user want's
to control batching himself (by @shargan)
Minor changes:
- Add
timestamp
field to produced message's metadata. This is needed to find
LOG_APPEND_TIME configured timestamps. Consumer.seek()
and similar API's now raise properValueError
's on
validation failure instead ofAssertionError
.
Bug fixes:
- Fix
connections_max_idle_ms
option, as earlier it was only applied to
bootstrap socket. (PR #299) - Fix
consumer.stop()
side effect of logging an exception
ConsumerStoppedError (issue #263) - Problem with Producer not able to recover from broker failure (issue #267)
- Traceback containing duplicate entries due to exception sharing (PR #247
by @Artimi) - Concurrent record consumption rasing
InvalidStateError('Exception is not set.')
(PR #249 by @aerkert) - Don't fail
GroupCoordinator._on_join_prepare()
ifcommit_offset()
throws exception (PR #230 by @shargan) - Send session_timeout_ms to GroupCoordinator constructor (PR #229 by @shargan)
Big thanks to:
- @shargan for Producer speed enhancements and the batch produce API
proposal/implementation. - @vineet-rh and other contributors for constant feedback on Consumer
problems, leading to the refactor mentioned above.
Minor bugfix release
Major docs upgrade. Many consumer fixes including new API's.
Starting from 0.3.0 we discourage imports from kafka
namespace. Starting from 0.4.0
aiokafka will stop supporting objects imported from kafka
namespace. Please import from aiokafka
namespaces instead.
- Moved all public structures and errors to
aiokafka
namespace. You will no
longer need to import fromkafka
namespace. - Changed ConsumerRebalanceListener to support either function or coroutine
foron_partitions_assigned
andon_partitions_revoked
callbacks. (PR #190
by @ask) - Added support for
offsets_for_times
,beginning_offsets
,end_offsets
API's. (issue #164) - Coordinator requests are now sent using a separate socket. Fixes slow commit
issue. (issuer #137, issue #128) - Added
seek_to_end
,seek_to_beginning
API's. (issue #154) - Updated documentation to provide more useful usage guide on both Consumer and
Producer interface.
Fix for Producer to handle retries correctly on node failures
- Fixed retry problem in Producer, when buffer is not reset to 0 offset.
Thanks to @ngavrysh for the fix in Tubular/aiokafka fork. (issue #184) - Fixed how Producer handles retries on Leader node failure. It just did not
work before... Thanks to @blugowski for the help in locating the problem.
(issue #176, issue #173) - Fixed degrade in v0.2.2 on Consumer with no group_id. (issue #166)
More bugfix for Consumer and reconnects
- Force reconnect after KafkaTimeoutException. (PR #149 by @Artimi)
- Fixed compacted topic handling. It could skip messages if some of them were
compacted (issue #71) - Fixed old issue with new topics not visible in pattern subscription (issue #46)
- Another fix for Consumer race condition on JoinGroup. This forces Leader to
wait for new metadata before assigning partitions. (issue #118) - Changed metadata listener in Coordinator to avoid 2 rejoins in a rare
condition (issue #108) getmany
will no longer return 0 results before timeout. (issue #117)
Big thanks to @Artimi for pointing out several of those issues.
Consumer enhancements and coordinator bugfix.
- Add a check to wait topic autocreation in Consumer, instead of raising UnknownTopicOrPartitionError (PR #92 by fabregas)
- Consumer now stops consumption after
consumer.stop()
call. Any newget*
calls will result in ConsumerStoppedError (PR #81) - Added
exclude_internal_topics
option for Consumer (PR #111) - Better support for pattern subscription when used with
group_id
(part of PR #111) - Fix for Consumer
subscribe
and JoinGroup race condition (issue #88). Coordinator will now notice subscription changes during rebalance and will join group again. (PR #106) - Changed logging messages according to KAFKA-3318. Now INFO level should be less messy and more informative. (PR #110)
- Add support for
connections_max_idle_ms
config (PR #113)
SSL support and bugfixes
- Added SSL support. (PR #81 by Drizzt1991)
- Fixed UnknownTopicOrPartitionError error on first message for autocreated topic (PR #96 by fabregas)
- Fixed
next_record
recursion (PR #94 by fabregas) - Fixed Heartbeat fail if no consumers (PR #92 by fabregas)
- Added docs addressing kafka-python and aiokafka differences (PR #70 by Drizzt1991)
- Added
max_poll_records
option for Consumer (PR #72 by Drizzt1991) - Fix kafka-python typos in docs (PR #69 by jeffwidman)
- Topics and partitions are now randomized on each Fetch request (PR #66 by Drizzt1991)
Updated Kafka to 0.10.0.1 and kafka-python to 1.3.1
- Bumped python-kafka version to 1.3.1 and Kafka to 0.10.1.0.
- Fixed auto version detection, to correctly handle 0.10.0.0 version
- Updated Fetch and Produce requests to use v2 with v0.10.0 message format on brokers.
This allows atimestamp
to be associated with messages. - Changed lz4 compression framing, as it was changed due to KIP-57 in new message format.
- Minor refactorings
Big thanks to @fabregas for the hard work on this release (PR #60)
Minor fixes
- Fixed bug with infinite loop on heartbeats with autocommit=True. #44
- Bumped python-kafka to version 1.1.1
- Fixed docker test runner with multiple interfaces
- Minor documentation fixes
v0.1.2
- Added Python3.5 usage example to docs
- Don't raise retryable exceptions in 3.5's async for iterator
- Fix Cancellation issue with producer's
send_and_wait
method