Skip to content

rust-v0.17.3
Compare
Choose a tag to compare
@rtyler rtyler released this 01 May 23:39
· 347 commits to main since this release
rust-v0.17.3
964c6b3

rust-v0.17.3 (2024-05-01)

Full Changelog

Implemented enhancements:

  • Limit concurrent ObjectStore access to avoid resource limitations in constrained environments #2457
  • How to get a DataFrame in Rust? #2404
  • Allow checkpoint creation when partion column is "timestampNtz " #2381
  • is there a way to make writing timestamp_ntz optional #2339
  • Update arrow dependency #2328
  • Release GIL in deltalake.write_deltalake #2234
  • Unable to retrieve custom metadata from tables in rust #2153
  • Refactor commit interface to be a Builder #2131

Fixed bugs:

  • Handle rate limiting during write contention #2451
  • regression : delta.logRetentionDuration don't seems to be respected #2447
  • Issue writing to mounted storage in AKS using delta-rs library #2445
  • TableMerger - when_matched_delete() fails when Column names contain special characters #2438
  • Generic DeltaTable error: External error: Arrow error: Invalid argument error: arguments need to have the same data type - while merge data in to delta table #2423
  • Merge on predicate throw error on date colum: Unable to convert expression to string #2420
  • Writing Tables with Append mode errors if the schema metadata is different #2419
  • Logstore issues on AWS Lambda #2410
  • Datafusion timestamp type doesn't respect delta lake schema #2408
  • Compacting produces smaller row groups than expected #2386
  • ValueError: Partition value cannot be parsed from string. #2380
  • Very slow s3 connection after 0.16.1 #2377
  • Merge update+insert truncates a delta table if the table is big enough #2362
  • Do not add readerFeatures or writerFeatures keys under checkpoint files if minReaderVersion or minWriterVersion do not satisfy the requirements #2360
  • Create empty table failed on rust engine #2354
  • Getting error message when running in lambda: message: "Too many open files" #2353
  • Temporary files filling up _delta_log folder - increasing table load time #2351
  • compact fails with merged schemas #2347
  • Cannot merge into table partitioned by date type column on 0.16.3 #2344
  • Merge breaks using logical datatype decimal128 #2343
  • Decimal types are not checked against max precision/scale at table creation #2331
  • Merge update+insert truncates a delta table #2320
  • Extract add.stats_parsed with wrong type #2312
  • Process fails without error message when executing merge #2310
  • delta_rs don't seems to respect the row group size #2309
  • Auth error when running inside VS Code #2306
  • Unable to read deltatables with binary columns: Binary is not supported by JSON #2302
  • Schema evolution not coercing with Large arrow types #2298
  • Panic in deltalake_core::kernel::snapshot::log_segment::list_log_files_with_checkpoint::{{closure}} #2290
  • Checkpoint does not preserve reader and writer features for the table protocol. #2288
  • Z-Order with larger dataset resulting in memory error #2284
  • Successful writes return error when using concurrent writers #2279
  • Rust writer should raise when decimal types are incompatible (currently writers and puts table in invalid state) #2275
  • Generic DeltaTable error: Version mismatch with new schema merge functionality in AWS S3 #2262
  • DeltaTable is not resilient to corrupted checkpoint state #2258
  • Inconsistent units of time #2256
  • Partition column comparison is an assertion rather than if block with raise exception #2242
  • Unable to merge column names starting from numbers #2230
  • Merging to a table with multiple distinct partitions in parallel fails #2227
  • cleanup_metadata not respecting custom logRetentionDuration #2180
  • Merge predicate fails with a field with a space #2167
  • When_matched_update causes records to be lost with explicit predicate #2158
  • Merge execution time grows exponetially with the number of column #2107
  • _internal.DeltaError when merging #2084
Assets 2
Loading