Skip to content

Releases: delta-io/delta-rs

python-v0.9.0

06 May 23:02
@wjones127 wjones127
python-v0.9.0
395d48b
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
python-v0.9.0

What's Changed

New features

Fixes

New Contributors

Full Changelog: python-v0.8.1...python-v0.9.0

Contributors

ColeMurray, nohajc, and 8 other contributors
Assets 2
Loading

rust-v0.10.0

03 May 00:26
@rtyler rtyler
rust-v0.10.0
edf1355
Compare
Choose a tag to compare
rust-v0.10.0

Full Changelog

Implemented enhancements:

  • Support Optimize on non-append-only tables #1125

Fixed bugs:

  • DataFusion integration incorrectly handles partition columns defined "first" in schema #1168
  • Datafusion: SQL projection returns wrong column for partitioned data #1292
  • Unable to query partitioned tables #1291

Merged pull requests:

  • chore: add deprecation notices for commit logic on DeltaTable #1323 (roeap)
  • fix: handle local paths on windows #1322 (roeap)
  • fix: scan partitioned tables with datafusion #1303 (roeap)
  • fix: allow special characters in storage prefix #1311 (wjones127)
  • feat: upgrade to Arrow 37 and Datafusion 23 #1314 (rtyler)
  • Hide the parquet/json feature behind our own JSON feature #1307 (rtyler)
  • Enable the json feature for the parquet crate #1300 (rtyler)
Assets 2
Loading

rust-v0.9.0

14 Apr 18:05
@rtyler rtyler
rust-v0.9.0
d7eb60b
Compare
Choose a tag to compare
rust-v0.9.0

Full Changelog

Implemented enhancements:

  • hdfs support #300
  • Add decimal primitive type to document #1280
  • Improve error message when filtering on non-existant partition columns #1218

Fixed bugs:

  • Datafusion table provider: issues with timestamp types #441
  • Not matching column names when creating a RecordBatch from MapArray #1257
  • All stores created using DeltaObjectStore::new have an identical object_store_url #1188

Merged pull requests:

  • Upgrade datafusion to 22 which brings arrow upgrades with it #1249 (rtyler)
  • chore: df / arrow changes after update #1288 (roeap)
  • feat: read schema from parquet files in datafusion scans #1266 (roeap)
  • HDFS storage support via datafusion-objectstore-hdfs #1279 (iajoiner)
  • Add description of decimal primitive to SchemaDataType #1281 (ognis1205)
  • Fix names and nullability when creating RecordBatch from MapArray #1258 (balbok0)
  • Simplify the Store Backend Configuration code #1265 (mrjoe7)
  • feat: optimistic transaction protocol #632 (roeap)
  • Write support for additional Arrow datatypes #1044(chitralverma)
  • Unique delta object store url #1212 (gruuya)
  • improve err msg on use of non-partitioned column #1221 (marijncv)
Assets 2
Loading

python-v0.8.1

13 Mar 23:59
@wjones127 wjones127
python-v0.8.1
8b85469
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
python-v0.8.1

What's Changed

Full Changelog: python-v0.8.0...python-v0.8.1

Contributors

wjones127 and gruuya
Assets 2
Loading

rust-v0.8.0

12 Mar 02:34
@wjones127 wjones127
rust-v0.8.0
6e96d8b
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
rust-v0.8.0

Full Changelog

Implemented enhancements:

  • feat(rust): support additional types for partition values #1170

Fixed bugs:

  • File pruning does not occur on partition columns #1175
  • Bug: Error loading Delta table locally #1157
  • Deltalake 0.7.0 with s3 feature compliation error due to rusoto_dynamodb version conflict #1191
  • Writing from a Delta table scan using WriteBuilder fails due to missing object store #1186

Merged pull requests:

Assets 2
Loading

python-v0.8.0

12 Mar 02:52
@wjones127 wjones127
python-v0.8.0
6e96d8b
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
python-v0.8.0

What's Changed

New Contributors

Full Changelog: python-v0.7.0...python-v0.8.0

Contributors

houqp, guyrt, and 7 other contributors
Assets 2
Loading

rust-v0.7.0

12 Feb 20:15
@wjones127 wjones127
rust-v0.7.0
48f99ba
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
rust-v0.7.0

Full Changelog

Implemented enhancements:

  • Support FSCK REPAIR TABLE Operation #1092
  • Expose the Delta Log in a DataFrame that's easy for analysis #1031
  • Provide case-insensitive storage options in backend #999
  • Support local file path in CreateBuilder::with_location() #998
  • Save operational params in the same way with delta io #1054 (ismoshkov)

Fixed bugs:

  • DeltaTable DataFusion TableProvider does not support filter pushdown #1064
  • DeltaTable DataFusion scan does not prune files properly #1063
  • deltalake.DeltaTable constructor hangs in Jupyter #1093
  • Transaction log JSON formatting issue when writing data via Python bindings #1017
  • crates.io entry is missing link to rustdoc documentation #1076
  • URL Registered with ObjectStore registry is different from url in DeltaScan #1018
  • Not able to connect to Azure Storage with client id/secret #977
  • Deltalake 0.5 crate s3 feature dynamodb version mismatch #973
  • Overwrite mode does not work with Azure #939
  • Use Chrono without default features #914
  • cargo test does not run due to tls conflict #985
  • Azure SAS authorization fails with <AuthenticationErrorDetail>Signature fields not well formed. #910

Merged pull requests:

* This Changelog was automatically generated by github_changelog_generator

Assets 2
Loading

python-v0.7.0

26 Jan 02:51
@wjones127 wjones127
python-v0.7.0
cc922a6
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
python-v0.7.0

What's Changed

New Contributors

Full Changelog: python-v0.6.4...python-v0.7.0

Contributors

gyscos, johnbatty, and 12 other contributors
Assets 2
Loading

rust-v0.6.0

20 Dec 17:04
@wjones127 wjones127
rust-v0.6.0
4096466
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
rust-v0.6.0

What's Changed

New Contributors

Full Changelog: python-v0.6.4...rust-v0.6.0

Contributors

andrei-ionescu, iajoiner, and 4 other contributors
Assets 2
Loading

rust-v0.5.0

28 Nov 16:23
@houqp houqp
rust-v0.5.0
cf84886
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
rust-v0.5.0

What's Changed

  • Add max and min values to Statistics by @viirya in #327
  • Use WebIdentityProvider for DynamoDb client in k8s by @rusty-jules in #328
  • bump rust version in preparation for the next release by @houqp in #329
  • fix automated rust release CD job by @houqp in #326
  • add pandas keyword to python package metadata by @houqp in #325
  • expose update_incremental API to python binding by @houqp in #332
  • update python related docs by @houqp in #331
  • Upgrade arrow, parquet and datafusion by @Dandandan in #335
  • added warning message if the detected glibc version is < 2.28 by @Smurphy000 in #334
  • Convert scalar value to correct type based on arrow data type. by @viirya in #336
  • Fix consecutive checkpoints by @mosyp in #333
  • Fix new clippy warnings coming up in CI by @xianwill in #341
  • perform incremental update after transaction commit by @houqp in #343
  • Add timestamp handling to checkpoint writer by @xianwill in #340
  • Add clear table state in load_version when no checkpoint found. by @zijie0 in #347
  • Low level create table by @Smurphy000 in #342
  • pub DeltaTable method to retrieve table configurations by @Smurphy000 in #356
  • Modify partition_values field type in Add/Remove actions. by @zijie0 in #354
  • fix sleep workaround in checkpoint test by @houqp in #360
  • Modify get_files_by_partitions to use partition values by @zijie0 in #362
  • Fix get_latest_version returning version < 0. by @zijie0 in #364
  • fix typo in python release CI config by @houqp in #365
  • cache cargo builds by @houqp in #359
  • Add '.tmp' suffix to temporary file of prepared commit by @mosyp in #366
  • support partition value string deserialization for float/double/date by @houqp in #363
  • Implement atomic put_obj. by @zijie0 in #367
  • Make Format.options to be required field by @mosyp in #370
  • Allow filesystem backend put_obj to overwrite existing by @mosyp in #376
  • Wrap DeltaTransactionError with DeltaTableError. by @zijie0 in #374
  • Refactoring of black, isort, mypy tools usages into pyproject.toml by @fvaleye in #378
  • Implement consistent behavior in Windows with regard to swap parameter. by @zijie0 in #379
  • Merge Cargo.toml into pyproject.toml by @fvaleye in #381
  • Update datafusion and ballista links in README by @ei-grad in #382
  • Add sts assume role credentials provider for S3 by @mosyp in #383
  • Reuse table/storage instances in checkpoints by @mosyp in #384
  • additional error handling to atomic_rename by @Marnixvdb in #386
  • Upgrade to DataFusion 5.0 by @Dandandan in #389
  • added initial commit info on create method for a DeltaTable by @Smurphy000 in #387
  • Google cloud by @blogle in #355
  • Remove version param from create_checkpoint_from_table by @mosyp in #399
  • Implement delete_objs in fs and s3 storage backends. by @zijie0 in #395
  • Add examples for reading delta table with Rust API. by @zijie0 in #400
  • Update pyproject definition in pyproject.toml by @fvaleye in #405
  • Use tokio::fs::rename in put_obj. by @zijie0 in #403
  • Fix duplicates on update call by @mosyp in #398
  • Add a Makefile build task in the Python binding by @fvaleye in #410
  • Add implementation for load_with_datetime in Python package. by @zijie0 in #411
  • Add filesystem argument for reading DeltaTable in Python binding by @fvaleye in #414
  • Fix reading nullable action fields from parquet by @mosyp in #417
  • Ensures that all table schemas are of StructType by @blogle in #415
  • Gcs writer bugs by @blogle in #412
  • Add S3StorageOptions to allow configuring S3 backend explicitly by @xianwill in #418
  • Read a DeltaTable using a Data Catalog by @fvaleye in #419
  • Change checkpoint creation logs from info to debug by @mosyp in #423
  • Add LICENSE file in the Python binding and refer it in the pyproject by @fvaleye in #422
  • Audit action field optionality by @fvaleye in #380
  • Introduce DeltaConfig and tombstones retention policy by @mosyp in #420
  • [README] Replace the inactive rust-dataframe with polars by @sa- in #426
  • Bump arrow to 6.0.0-SNAPSHOT and bring map support to schema by @mosyp in #375
  • Support partition value string deserialization for timestamp/binary by @zijie0 in #371
  • Document the valid primitive types by @Ekleog in #430
  • Add is_non_acquirable field to the dynamodb lock by @mosyp in #429
  • Clean up DeltaTransactionError by @mosyp in #432
  • Optimize remove action apply with early iteration exit #424 by @akshay26031996 in #431
  • Decode path in Add and Remove actions. by @zijie0 in #434
  • reenable datafusion integration with temporary fork by @houqp in #436
  • Add history command in delta-rs by @fvaleye in #428
  • Release Python binding version 0.5.3 by @fvaleye in #439
  • Add delete_lock and fix release_lock by @mosyp in #440
  • Fixing test to compare sorted vec by @akshay26031996 in #443
  • Batch-apply remove actions in tombstone handling by @dispanser in #444
  • Update datafusion links by @bbigras in #446
  • Run all tests under s3 feature flag by @mosyp in #447
  • Add maturin develop command with extras in Python binding by @fvaleye in #448
  • README: mark Checkpoint creation as done for Rust by @bbigras in #449
  • Fix broken tombstones metadata when extended_file_metadata is different between tomstones in state by @mosyp in #450
  • No tombstone loading by @dispanser in #445
  • return lazy iterator in get tombstone methods by @houqp in #452
  • Generate new session name on assume role credentials provider refresh by @mosyp in #451
  • Add pool_idle_timeout options for s3 and sts clients by @mosyp in #458
  • Do action reconciliation by @viirya in #456
  • Use action default stats by @viirya in #459
  • Add new module for DeltaTableState by @viirya in #464
  • Support hash lookup by path string for Remove action by @viirya in #462
  • Fix coverage of the Python tests by @fvaleye in #467
  • materialize tables in pyhton via native storage backend by @roeap in #463
  • Make file storage backend's atomic rename async by @viirya in #471
  • Add GCS feature to the Python Cargo.toml file by @kelvins in #476
  • Throw an error when filter key is not in partitioned columns. by @zijie0 in #475
  • Fix documentation for the DeltaStorageHandler by @fvaleye in #483
  • Update README.adoc by @dennyglee in #482
  • Update az...
Read more

Contributors

bbigras, viirya, and 40 other contributors
Assets 2
Loading