Release 0.5.0

Co-authored-by: Michał Bartoszkiewicz <embe@pathway.com>
Co-authored-by: Jan Chorowski <janek@pathway.com>
Co-authored-by: Xavier Gendre <xavier@pathway.com>
Co-authored-by: Adrian Kosowski <adrian@pathway.com>
Co-authored-by: Jakub Kowalski <kuba@pathway.com>
Co-authored-by: Sergey Kulik <sergey@pathway.com>
Co-authored-by: Mateusz Lewandowski <mateusz@pathway.com>
Co-authored-by: Mohamed Malhou <mohamed@pathway.com>
Co-authored-by: Krzysztof Nowicki <krzysiek@pathway.com>
Co-authored-by: Richard Pelgrim <richard.pelgrim@pathway.com>
Co-authored-by: Kamil Piechowiak <kamil@pathway.com>
Co-authored-by: Paweł Podhajski <pawel.podhajski@pathway.com>
Co-authored-by: Olivier Ruas <olivier@pathway.com>
Co-authored-by: Przemysław Uznański <przemek@pathway.com>
Co-authored-by: Sebastian Włudzik <sebastian.wludzik@pathway.com>
GitOrigin-RevId: f84bb429efb65b7b7a8612921a8c58f8c0f6f7e3

Author: 16 people

.github/workflows/release.yml

@@ -62,6 +62,15 @@ jobs:
           rm -f LICENSE_*-LICENSE-*
           for filename in library_licenses/*; do cp "$filename" "LICENSE_$(basename "${filename}")"; done;
 
+      - name: Set package version
+        id: set-version
+        run: |
+          PACKAGE_VERSION=$(perl -nle 'print $& while m{^version[[:space:]]*=[[:space:]]"\K[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+(?=")}g' Cargo.toml)
+          [[ -z "$PACKAGE_VERSION" ]] && { echo "Malformed package version in Cargo.toml" ; exit 1; }
+
+          echo $PACKAGE_VERSION
+          echo "__version__ = \"${PACKAGE_VERSION}\"" > python/pathway/internals/version.py
+
       - name: Build package Ubuntu x86-x64
         if: ${{ matrix.os == 'ubuntu-22.04'}}
         uses: PyO3/maturin-action@v1
@@ -147,7 +156,7 @@ jobs:
 
       - name: Create dir for wheels
         run: |
-          mkdir wheels
+          mkdir -p wheels
             
       - uses: actions/download-artifact@master
         with:
@@ -183,7 +192,7 @@ jobs:
     steps:
       - name: Create dir for wheels
         run: |
-          mkdir wheels
+          mkdir -p wheels
 
       - uses: actions/download-artifact@master
         if: ${{ matrix.os == 'ec2-macOS' }}
@@ -200,13 +209,12 @@ jobs:
       - name: Install and verify ${{ matrix.os }} package
         run: |
           set -ex
-          # Pathway http monitoring set port
-          source .github/workflows/bash_scripts/PATHWAY_MONITORING_HTTP_PORT.sh
           ENV_NAME="testenv_${{ matrix.python-version }}"
           rm -rf "${ENV_NAME}"
           python"${{ matrix.python-version }}" -m venv "${ENV_NAME}"
           source "${ENV_NAME}/bin/activate"
           WHEEL=(./wheels/pathway-*.whl)
+          PATHWAY_MONITORING_HTTP_PORT=20099
           pip install --prefer-binary "${WHEEL}[tests]"
           # --confcutdir anything below to avoid picking REPO_TOP_DIR/conftest.py
           python -m pytest --confcutdir "${ENV_NAME}" --doctest-modules --pyargs pathway
@@ -241,7 +249,7 @@ jobs:
 
       - name: Create dir for wheels
         run: |
-          mkdir wheels
+          mkdir -p wheels
 
       - uses: actions/download-artifact@master
         with:

CHANGELOG.md

@@ -6,6 +6,30 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 ## [Unreleased]
 
+## [0.5.0] - 2023-10-04
+
+### Added
+- `Schema` method `typehints` that returns dict of mypy-compatible typehints.
+
+### Changed
+- **BREAKING**: renamed `Table` method `dtypes` to `typehints`. It now returns a `dict` of mypy-compatible typehints.
+- **BREAKING**: `Schema.__getitem__` returns a data class `ColumnSchema` containing all related information on particular column.
+
+### Added
+- Support for JSON parsing from CSV sources.
+- `restrict` method in `Table` to restrict table universe to the universe of the other table.
+- Better support for postgresql types in the output connector.
+
+### Changed
+- **BREAKING**: `tuple` reducer used after intervals_over window now sorts values by time.
+- **BREAKING**: expressions used in `select`, `filter`, `flatten`, `with_columns`, `with_id`, `with_id_from` have to have the same universe as the table. Earlier it was possible to use an expression from a superset of a table universe. To use expressions from wider universes, one can use `restrict` on the expression source table.
+- **BREAKING**: `pw.universes.promise_are_equal(t1, t2)` no longer allows to use references from `t1` and `t2` in a single expression. To change the universe of a table, use `with_universe_of`.
+- **BREAKING**: `ix` and `ix_ref` are temporarily broken inside joins (both temporal and ordinary).
+- `select`, `filter`, `concat` keep columns as a single stream. The work for other operators is ongoing.
+
+### Fixed
+- Optional types other than string correctly output to PostgreSQL.
+
 ## [0.4.1] - 2023-09-25
 
 ### Added
@@ -16,6 +40,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Added
 - Support for JSON data format, including `pw.Json` type.
 - Methods `as_int()`, `as_float()`, `as_str()`, `as_bool()` to convert values from `Json`.
+- New argument `skip_nones` for `tuple` and `sorted_tuple` reducers.
+- New argument `is_outer` for `intervals_over` window.
+- `pw.schema_from_dict` and `pw.schema_from_csv` for generating schema based, respectively, on provided definition as a dictionary and CSV file with sample data.
+- `generate_class` method in `Schema` class for generating schema class code.
 
 ### Changed
 - Method `get()` and `[]` to support accessing elements in Jsons.