Skip to content

KSML 0.8.0 - Major milestone for KSML
Compare
Choose a tag to compare
@richard-axual richard-axual released this 08 Mar 12:48
· 73 commits to main since this release
0.8.0
1ced320

We are happy to announce the release of KSML 0.8.0!

The 0.8.0 release was the next big step for KSML, providing an updated syntax for KSML, increased stability and a lot of exciting features.
See the change and feature list below for all the changes made!

  • Reworked all parsing logic, to allow for exporting the JSON schema of the KSML specification:
    • docs/specification.md is now derived from internal parser logic, guaranteeing consistency and completeness.
    • examples/ksml.json contains the JSON schema, which can be loaded into IDEs for syntax validation and completion.
  • Improved schema handling:
    • Better compatibility checking between schema fields.
  • Improved support for state stores:
    • Update to state store typing and handling.
    • Manual state stores can be defined and referenced in pipelines.
    • Manual state stores are also available in Python functions.
    • State stores can be used 'side-effect-free' (eg. no AVRO schema registration)
  • Python function improvements:
    • Automatic variable assignment for state stores.
    • Every Python function can use a Java Logger, integrating Python output with KSML log output.
    • Type inference in situations where parameters or result types can be derived from the context.
  • Lots of small language updates:
    • Improve readability for store types, filter operations and windowing operations
    • Introduction of the "as" operation, which allows for pipeline referencing and chaining.
  • Better data type handling:
    • Separation of data types and KSML core, allowing for easier addition of new data types in the future.
    • Automatic conversion of data types, removing common pipeline failure scenarios.
    • New implementation for CSV handling.
  • Merged the different runners into a single runner.
    • KSML definitions can now include both producers (data generators) and pipelines (Kafka Streams topologies).
    • Removal of Kafka and Axual backend distinctions.
  • Configuration file updates, allowing for running multiple definitions in a single runner (each in its own namespace).
  • Examples updated to reflect the latest definition format.
  • Documentation updated.

Full Changelog: 0.2.2...0.8.0

Assets 2
Loading