Releases: Bendabir/torch-kafka
Releases · Bendabir/torch-kafka
Make records skippable by the dataset
Easier consumer instanciation override
Minor correction to ease the override of consumer parameters when instanciated.
Python package
This release addresses the following items :
- The module can now be installed as a regular package. You can use
pip install git+https://github.com/bendabir/torch-kafka@1.1.0
to do so. - Fixed possible deadlocks when committing offsets using multiprocessing.
Minor fixes
This fixes some possible minor issues :
- signal handler used to commit the workers is now reset to default handler once iteration is over.
auto_commit
helper now works on all datasets and has no effect on datasets that are not instances of KafkaDataset.
Initial release
This first release provide a PyTorch Dataset for Kafka and a method to auto-commit the data that are generated by a PyTorch DataLoader. It supports both single-processing and multi-processing.
Supported system and versions are :
- Linux (tested with Ubuntu 18.04);
- Python 3.8+;
- PyTorch 1.6+;
- kafka-python 2.0+.