From a617c14d2ae6fd21a7323d3e0912b3877aeb7949 Mon Sep 17 00:00:00 2001 From: Levi Tamasi Date: Fri, 20 Sep 2024 17:09:41 -0700 Subject: [PATCH] Update HISTORY for the 9.7 release --- HISTORY.md | 20 +++++++++++++++++++ ...blob_garbage_collection_force_threshold.md | 1 - unreleased_history/behavior_changes/db_id.md | 1 - .../behavior_changes/fifo-temp-compaction.md | 1 - .../ingest-live-file-with-move.md | 1 - .../behavior_changes/link-file-ingest.md | 1 - .../sst_file_manager_untrack_close.md | 2 -- .../bug_fixes/bug-refit-level.md | 1 - .../bug_fixes/memtable_prefix.md | 1 - .../bug_fixes/wal-error-recovery.md | 1 - .../new_features/customizable_cache.md | 1 - .../new_features/prefix_seek_opt_in_only.md | 1 - .../new_features/tp_largest_seqno.md | 1 - 13 files changed, 20 insertions(+), 13 deletions(-) delete mode 100644 unreleased_history/behavior_changes/blob_garbage_collection_force_threshold.md delete mode 100644 unreleased_history/behavior_changes/db_id.md delete mode 100644 unreleased_history/behavior_changes/fifo-temp-compaction.md delete mode 100644 unreleased_history/behavior_changes/ingest-live-file-with-move.md delete mode 100644 unreleased_history/behavior_changes/link-file-ingest.md delete mode 100644 unreleased_history/behavior_changes/sst_file_manager_untrack_close.md delete mode 100644 unreleased_history/bug_fixes/bug-refit-level.md delete mode 100644 unreleased_history/bug_fixes/memtable_prefix.md delete mode 100644 unreleased_history/bug_fixes/wal-error-recovery.md delete mode 100644 unreleased_history/new_features/customizable_cache.md delete mode 100644 unreleased_history/new_features/prefix_seek_opt_in_only.md delete mode 100644 unreleased_history/new_features/tp_largest_seqno.md diff --git a/HISTORY.md b/HISTORY.md index f3ad192da68..d4fc843a2bc 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -1,6 +1,26 @@ # Rocksdb Change Log > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt` +## 9.7.0 (09/20/2024) +### New Features +* Make Cache a customizable class that can be instantiated by the object registry. +* Add new option `prefix_seek_opt_in_only` that makes iterators generally safer when you might set a `prefix_extractor`. When `prefix_seek_opt_in_only=true`, which is expected to be the future default, prefix seek is only used when `prefix_same_as_start` or `auto_prefix_mode` are set. Also, `prefix_same_as_start` and `auto_prefix_mode` now allow prefix filtering even with `total_order_seek=true`. +* Add a new table property "rocksdb.key.largest.seqno" which records the largest sequence number of all keys in file. It is verified to be zero during SST file ingestion. + +### Behavior Changes +* Changed the semantics of the BlobDB configuration option `blob_garbage_collection_force_threshold` to define a threshold for the overall garbage ratio of all blob files currently eligible for garbage collection (according to `blob_garbage_collection_age_cutoff`). This can provide better control over space amplification at the cost of slightly higher write amplification. +* Set `write_dbid_to_manifest=true` by default. This means DB ID will now be preserved through backups, checkpoints, etc. by default. Also add `write_identity_file` option which can be set to false for anticipated future behavior. +* In FIFO compaction, compactions for changing file temperature (configured by option `file_temperature_age_thresholds`) will compact one file at a time, instead of merging multiple eligible file together (#13018). +* Support ingesting db generated files using hard link, i.e. IngestExternalFileOptions::move_files/link_files and IngestExternalFileOptions::allow_db_generated_files. +* Add a new file ingestion option `IngestExternalFileOptions::link_files` to hard link input files and preserve original files links after ingestion. +* DB::Close now untracks files in SstFileManager, making avaialble any space used +by them. Prior to this change they would be orphaned until the DB is re-opened. + +### Bug Fixes +* Fix a bug in CompactRange() where result files may not be compacted in any future compaction. This can only happen when users configure CompactRangeOptions::change_level to true and the change level step of manual compaction fails (#13009). +* Fix handling of dynamic change of `prefix_extractor` with memtable prefix filter. Previously, prefix seek could mix different prefix interpretations between memtable and SST files. Now the latest `prefix_extractor` at the time of iterator creation or refresh is respected. +* Fix a bug with manual_wal_flush and auto error recovery from WAL failure that may cause CFs to be inconsistent (#12995). The fix will set potential WAL write failure as fatal error when manual_wal_flush is true, and disables auto error recovery from these errors. + ## 9.6.0 (08/19/2024) ### New Features * *Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush. diff --git a/unreleased_history/behavior_changes/blob_garbage_collection_force_threshold.md b/unreleased_history/behavior_changes/blob_garbage_collection_force_threshold.md deleted file mode 100644 index 0c4b8bba25c..00000000000 --- a/unreleased_history/behavior_changes/blob_garbage_collection_force_threshold.md +++ /dev/null @@ -1 +0,0 @@ -Changed the semantics of the BlobDB configuration option `blob_garbage_collection_force_threshold` to define a threshold for the overall garbage ratio of all blob files currently eligible for garbage collection (according to `blob_garbage_collection_age_cutoff`). This can provide better control over space amplification at the cost of slightly higher write amplification. diff --git a/unreleased_history/behavior_changes/db_id.md b/unreleased_history/behavior_changes/db_id.md deleted file mode 100644 index 5536f3ce525..00000000000 --- a/unreleased_history/behavior_changes/db_id.md +++ /dev/null @@ -1 +0,0 @@ -* Set `write_dbid_to_manifest=true` by default. This means DB ID will now be preserved through backups, checkpoints, etc. by default. Also add `write_identity_file` option which can be set to false for anticipated future behavior. diff --git a/unreleased_history/behavior_changes/fifo-temp-compaction.md b/unreleased_history/behavior_changes/fifo-temp-compaction.md deleted file mode 100644 index 7ba5f643d4f..00000000000 --- a/unreleased_history/behavior_changes/fifo-temp-compaction.md +++ /dev/null @@ -1 +0,0 @@ -* In FIFO compaction, compactions for changing file temperature (configured by option `file_temperature_age_thresholds`) will compact one file at a time, instead of merging multiple eligible file together (#13018). \ No newline at end of file diff --git a/unreleased_history/behavior_changes/ingest-live-file-with-move.md b/unreleased_history/behavior_changes/ingest-live-file-with-move.md deleted file mode 100644 index 444a7a45e7f..00000000000 --- a/unreleased_history/behavior_changes/ingest-live-file-with-move.md +++ /dev/null @@ -1 +0,0 @@ -* Support ingesting db generated files using hard link, i.e. IngestExternalFileOptions::move_files/link_files and IngestExternalFileOptions::allow_db_generated_files. \ No newline at end of file diff --git a/unreleased_history/behavior_changes/link-file-ingest.md b/unreleased_history/behavior_changes/link-file-ingest.md deleted file mode 100644 index e9f909eee73..00000000000 --- a/unreleased_history/behavior_changes/link-file-ingest.md +++ /dev/null @@ -1 +0,0 @@ -* Add a new file ingestion option `IngestExternalFileOptions::link_files` to hard link input files and preserve original files links after ingestion. \ No newline at end of file diff --git a/unreleased_history/behavior_changes/sst_file_manager_untrack_close.md b/unreleased_history/behavior_changes/sst_file_manager_untrack_close.md deleted file mode 100644 index 7251b547e71..00000000000 --- a/unreleased_history/behavior_changes/sst_file_manager_untrack_close.md +++ /dev/null @@ -1,2 +0,0 @@ -DB::Close now untracks files in SstFileManager, making avaialble any space used -by them. Prior to this change they would be orphaned until the DB is re-opened. diff --git a/unreleased_history/bug_fixes/bug-refit-level.md b/unreleased_history/bug_fixes/bug-refit-level.md deleted file mode 100644 index f41e699e6a1..00000000000 --- a/unreleased_history/bug_fixes/bug-refit-level.md +++ /dev/null @@ -1 +0,0 @@ -* Fix a bug in CompactRange() where result files may not be compacted in any future compaction. This can only happen when users configure CompactRangeOptions::change_level to true and the change level step of manual compaction fails (#13009). \ No newline at end of file diff --git a/unreleased_history/bug_fixes/memtable_prefix.md b/unreleased_history/bug_fixes/memtable_prefix.md deleted file mode 100644 index d7b45c65ee5..00000000000 --- a/unreleased_history/bug_fixes/memtable_prefix.md +++ /dev/null @@ -1 +0,0 @@ -* Fix handling of dynamic change of `prefix_extractor` with memtable prefix filter. Previously, prefix seek could mix different prefix interpretations between memtable and SST files. Now the latest `prefix_extractor` at the time of iterator creation or refresh is respected. diff --git a/unreleased_history/bug_fixes/wal-error-recovery.md b/unreleased_history/bug_fixes/wal-error-recovery.md deleted file mode 100644 index 07d87473d60..00000000000 --- a/unreleased_history/bug_fixes/wal-error-recovery.md +++ /dev/null @@ -1 +0,0 @@ -* Fix a bug with manual_wal_flush and auto error recovery from WAL failure that may cause CFs to be inconsistent (#12995). The fix will set potential WAL write failure as fatal error when manual_wal_flush is true, and disables auto error recovery from these errors. \ No newline at end of file diff --git a/unreleased_history/new_features/customizable_cache.md b/unreleased_history/new_features/customizable_cache.md deleted file mode 100644 index b50d03c249e..00000000000 --- a/unreleased_history/new_features/customizable_cache.md +++ /dev/null @@ -1 +0,0 @@ -Make Cache a customizable class that can be instantiated by the object registry. diff --git a/unreleased_history/new_features/prefix_seek_opt_in_only.md b/unreleased_history/new_features/prefix_seek_opt_in_only.md deleted file mode 100644 index 71c3cd68936..00000000000 --- a/unreleased_history/new_features/prefix_seek_opt_in_only.md +++ /dev/null @@ -1 +0,0 @@ -* Add new option `prefix_seek_opt_in_only` that makes iterators generally safer when you might set a `prefix_extractor`. When `prefix_seek_opt_in_only=true`, which is expected to be the future default, prefix seek is only used when `prefix_same_as_start` or `auto_prefix_mode` are set. Also, `prefix_same_as_start` and `auto_prefix_mode` now allow prefix filtering even with `total_order_seek=true`. diff --git a/unreleased_history/new_features/tp_largest_seqno.md b/unreleased_history/new_features/tp_largest_seqno.md deleted file mode 100644 index 6776cfcf234..00000000000 --- a/unreleased_history/new_features/tp_largest_seqno.md +++ /dev/null @@ -1 +0,0 @@ -* Add a new table property "rocksdb.key.largest.seqno" which records the largest sequence number of all keys in file. It is verified to be zero during SST file ingestion. \ No newline at end of file