Skip to content

v0.7.0-incubating

Latest
Latest
Compare
Choose a tag to compare
@zhoujinsong zhoujinsong released this 02 Aug 04:06
· 137 commits to master since this release
v0.7.0-incubating
d8e2784

Highlight:

  • Support automatically generating and managing Tags and Branches for tables #1354
  • Support killing the running optimizing process #1862
  • Support detail page for optimizing process #2114
  • Support Spark-based optimizer #1812
  • Add metric collection for AMS #1528
  • Support managing tables with multiple formats under one Catalog #1061

Features:

  • Support automatically generating and managing Tags and Branchs for Table #1354
  • Support killing the running optimizing process #1862
  • Support detail page for Optimizing Process #2114
  • Support Spark-based optimizer #1812
  • Add metric collection for ams #1528
  • Support managing tables with multiple formats under one Catalog #1061
  • Make local terminal cores configurable #2492
  • Support reading encrypted iceberg data files #2401
  • Add basic authentication to REST API #2667

Improvements:

  • Improve major optimizing: compact segment files to target size #2330
  • Put the database username and password into Kubernetes Secrets #2291
  • Avoid releasing external optimizer instances through ams #2315
  • Mixed-format Table stats for Trino engine need to be calculated individually #2246
  • Add planning optimizing status for table #2290
  • Support computed columns for mixed-format tables in Flink DDL #1457
  • The Files page supports filtering by partition name and sorting by dictionary value. #2316
  • Support log out of the ams dashboard #2359
  • Support parallelized planning in one optimizer group #1951
  • Skip cleaning up dangling delete files for Iceberg V1 table #2222
  • Configure iceberg.worker.num-threads in the configuration file #2386
  • Support sorting the external Iceberg table list #1716
  • Add table filter for catalog properties #2310
  • Show the format version of Iceberg Table #2260
  • Make the maximum input file size for each optimizing thread configurable #2385
  • The Files page supports filtering by partition name and sorting by dictionary value. #2316
  • Use partition filter to speed up optimizing plan #2417
  • Add more configurable properties to the ams server #2333
  • Dispose of the current tasks after task deserialization error #2521
  • Paimon snapshot display on the page should be arranged in reverse order according to the snapshot time of the table. #2606
  • Add support for setting/altering mixed-format table properties with Flink SQL #2
  • Helm adds Kyuubi (as a terminal backend) and PostgreSQL-related conf #2306
  • Validate hive-site.xml when uploading a file for Hive-catalog #2520
  • Optimize the speed of searching for tables in the tables navigation bar #2215
  • Use CachingCatalog to reduce the time cost of IcebergCatalogWrapper#loadTable #1794
  • Enable return URL parameter when redirecting to login page #2637
  • Display more details on the page of Optimizing->Optimizers #2698
  • Improve the version info about the project #2813
  • AMS should use G1 GC by default #2830
  • Support different simple authorization user names for different catalogs #2918
  • Move admin username and password to Kubernetes secrets #2948
  • Add Tests for helm chart templates #2778
  • Support S3 storage for Paimon format catalog #2972
  • Support changing the language to Chinese for Dashboard #2958

BugFixes:

  • After disabled self-optimizing, the table status is not updated to idle #2250
  • Optimizing commit failed: org.apache.iceberg.exceptions.ValidationException: Missing required files to delete #2454
  • Periodic snapshots expiring tasks will be scheduled redundantly #2453
  • When using Flink API to write data into a mixed-iceberg table, it will update the table metadata to incorrect value #2525
  • Optimizing not trigged when partition value is null #2542
  • When expiring historical data, the latest snapshot generated by amoro should not be used #2555
  • The process of deleting expired snapshots was interrupted by exception #2579
  • Loading Mixed-Iceberg format tables of internal catalog failed in Flink #2587
  • Failed to remove orphan files after dropping a partition field #2617
  • The Last Commit Time for the Paimon partitioned table in the File pages has the wrong zone #2607
  • If an exception occurs in externalCatalog.listTables during the exploreExternalCatalog process, exploreExternalCatalog will fail this time. #2638
  • Failed to view snapshot details #2614
  • Hive table name may be too long for AMS table identifiers #2649
  • Skip the deleted directories when cleaning orphan files #2619
  • Optimizing process stuck after a few failures #2623
  • Lack of database type mandatory checking, leading to NPE in existing MySQL config check #2733
  • Iceberg temporary snapshot tables should be excluded from data cleanups #2488
  • Filformat ORC does not support rolling new files for mixed_format #2560
Assets 2
Loading