Revision release 1.0.4

What's Changed

• [Backport] Fix crash when using AVRO CharSequence encodings and nested objects by @jeroenvandisseldorp in #164

Full Changelog: 1.0.3...1.0.4

Revision release 1.0.3

What's Changed

• Fix excessive CPU Usage by @richard-axual in #157
• Update Avro dependency to 1.11.4 to fix CVE-2024-47561

Full Changelog: 1.0.2...1.0.3

Revision release 1.0.2

What's Changed

• Solve missing SR url properties error for apps without AVRO by @jeroenvandisseldorp in #145
• Fix transformtolist result type by @jeroenvandisseldorp in #146
• Serialization fixes by @jeroenvandisseldorp in #147
• Fix missing object type in language spec by @jeroenvandisseldorp in #148
• Separate liveness, readiness and startup services by @richard-axual in #150
• Fix logic issue with Prometheus Export and ServiceMonitor by @richard-axual in #152
• Upgrade to Kafka 3.8.0 by @jeroenvandisseldorp in #151
• Add namespace and name to Function log messages and exceptions by @richard-axual in #153

Full Changelog: 1.0.1...1.0.2

Revision release 1.0.1

What's Changed

• Fix issue Joins not working. No optimisation causes tables to have no data by @richard-axual in #136
• Update windowed types and small fixes by @jeroenvandisseldorp in #138
• Make sure the file system specific separator character is used. by @richard-axual in #139
• Add runtime dependencies to KSML image by @richard-axual in #142
• Fix Parse Error on Join/LeftJoin/OuterJoins by @richard-axual in #143
• Add proper stream data type flattening for transform operations by @jeroenvandisseldorp in #144

Full Changelog: 1.0.0...1.0.1

1.0.0

KSML 1.0.0 is released!!

We are very proud that KSML 1.0.0 is now released.
With this release we've introduced several features that we consider make KSML ready for production.
A big thanks to the main contributors, @jeroenvandisseldorp. @tonvanbart, @xsreality, @richard-axual and of course all users that provided us with feedback. Without you we couldn't have gotten here!

Where to get the images?

The images can be downloaded from Docker Hub using axual/ksml:1.0.0 or axual/ksml:latest but also from Axuals open source registry at registry.axual.io/opensource/images/axual/ksml:1.0.0 or registry.axual.io/opensource/images/axual/ksml:latest

Helm Charts are here!

The KSML Helm Charts make it easy to deploy a KSML application to Kubernetes.
KSML will run as a StatefulSet and uses a VolumeClaimTemplate to store state, allowing for scaling of your KSML app
It can use all KSML features, while also providing the possibility to configure Kubernetes properties like resource limits, node selectors, affinity, tolerations, and more.

The chart is hosted in the Axual OpenSource registry as well, use the following OCI url to access it oci://registry.axual.io/opensource/charts/ksml

You can use the helm command to get the information.

helm show readme oci://registry.axual.io/opensource/charts/ksml --version 1.0.0

Syntax changes and added operations

• The to sink with topicNameExtractor has been split into two separate operations, to and toTopicNameExtractor
• It is possible to set an offsetResetPolicy on a streams definition
• It is possible to extract timestamps from a message payload by defining a timestampExtractor on a stream

What's Changed

• Implement timestamp extractor for KSML Streams by @jeroenvandisseldorp in #128
• Fix NullPointer exception in PrometheusExport by @richard-axual in #131
• Use custom properties file to get version information by @richard-axual in #130
• Update documentation by @tonvanbart in #129
• Update metrics implementation by @richard-axual in #132
• Optimize syntax by @jeroenvandisseldorp in #133
• Update docs by @jeroenvandisseldorp in #134
• Added helm charts by @xsreality in #85

New Contributors

• @xsreality made their first contribution in #85

Full Changelog: 0.9.1...1.0.0

0.9.1

What's Changed

• Fix failing test in GitHub Actions during release by @tonvanbart in #126
• Unified build workflows by @tonvanbart in #127

Full Changelog: 0.9.0...0.9.1

0.9.0

What's Changed

• Collectable metrics
• New topology test suite
• Python context hardening
• Improved handling of Kafka tombstones
• Added flexibility to producers (single shot, n-shot, or user condition-based)
• JSON Logging support
• Bumped GraalVM to 23.1.2
• Bumped several dependency versions
• Several fixes and security updates

Full Changelog: 0.8.0...0.9.0

Bugfix release 0.2.4

What's Changed

• Fix for conversion of DataUnion to native Java type by @jeroenvandisseldorp in #117
• Release 0.2.4 by @jeroenvandisseldorp in #121

Full Changelog: 0.2.3...0.2.4

0.2.3

What's Changed

• Fix crashing on Null keys and values for 0.2 release by @richard-axual in #109
• Fix the bugfix by @jeroenvandisseldorp in #111
• Release 0.2.3 take two by @jeroenvandisseldorp in #114

Full Changelog: 0.2.2...0.2.3

KSML 0.8.0 - Major milestone for KSML

We are happy to announce the release of KSML 0.8.0!

The 0.8.0 release was the next big step for KSML, providing an updated syntax for KSML, increased stability and a lot of exciting features.
See the change and feature list below for all the changes made!

• Reworked all parsing logic, to allow for exporting the JSON schema of the KSML specification:
  • docs/specification.md is now derived from internal parser logic, guaranteeing consistency and completeness.
  • examples/ksml.json contains the JSON schema, which can be loaded into IDEs for syntax validation and completion.
• Improved schema handling:
  • Better compatibility checking between schema fields.
• Improved support for state stores:
  • Update to state store typing and handling.
  • Manual state stores can be defined and referenced in pipelines.
  • Manual state stores are also available in Python functions.
  • State stores can be used 'side-effect-free' (eg. no AVRO schema registration)
• Python function improvements:
  • Automatic variable assignment for state stores.
  • Every Python function can use a Java Logger, integrating Python output with KSML log output.
  • Type inference in situations where parameters or result types can be derived from the context.
• Lots of small language updates:
  • Improve readability for store types, filter operations and windowing operations
  • Introduction of the "as" operation, which allows for pipeline referencing and chaining.
• Better data type handling:
  • Separation of data types and KSML core, allowing for easier addition of new data types in the future.
  • Automatic conversion of data types, removing common pipeline failure scenarios.
  • New implementation for CSV handling.
• Merged the different runners into a single runner.
  • KSML definitions can now include both producers (data generators) and pipelines (Kafka Streams topologies).
  • Removal of Kafka and Axual backend distinctions.
• Configuration file updates, allowing for running multiple definitions in a single runner (each in its own namespace).
• Examples updated to reflect the latest definition format.
• Documentation updated.

Full Changelog: 0.2.2...0.8.0