MDEV-37070 per-index adaptive hash index parameters #4507
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
Contributor
Author
|
|
Contributor
|
Yes, the combinations I'm tuning the test. diff --git a/mysql-test/suite/innodb/t/index_ahi_option.test b/mysql-test/suite/innodb/t/index_ahi_option.test
index 5b86c2a250d..7c92eab581e 100644
--- a/mysql-test/suite/innodb/t/index_ahi_option.test
+++ b/mysql-test/suite/innodb/t/index_ahi_option.test
@@ -46,8 +46,8 @@ INSERT INTO t1 VALUES (50001, 50, 1, 1), (50002, 50, 2, 2),
--echo #
SET GLOBAL innodb_monitor_enable = module_adaptive_hash;
-let $warm_up_rounds= 200;
-let $query_rounds= 10;
+let $warm_up_rounds= 0;
+let $query_rounds= 140;
--echo # Warming up AHI
let $i = $warm_up_rounds;
@@ -194,7 +194,7 @@ let $val0 = $val1;
ALTER TABLE t1 adaptive_hash_index='ON';
--echo # Warming up AHI
-let $i = 200;
+let $i = $warm_up_rounds;
while ($i)
{
--disable_query_logI'll run again the tests with |
dr-m
commented
Jan 22, 2026
Contributor
Author
dr-m
left a comment
dr-m
left a comment
There was a problem hiding this comment.
It’s a great idea to combine several fields into a single 32-bit atomic field. But, some "pretty" code can actually result in some quite ugly machine code (with conditional branches).
storage/innobase/include/dict0mem.h
Outdated
| - bit 23: fields valid | ||
| - bit 24: bytes valid | ||
| - bit 25: left valid | ||
| - bits [26, 31]: spare (6 bits) |
Contributor
Author
There was a problem hiding this comment.
It could be better to allocate something useful (such as part of the enabled/disabled preference) at the most significant bit. For example, on x86, the test instruction would allow the most significant bit to be copied to a flag.
Contributor
There was a problem hiding this comment.
I kept this as is in the last version, since I did not want to change too much the new inline bool is_enabled(const dict_index_t *index) const noexcept logic. Is this acceptable?
also allow TRUE/FALSE for booleans
avoid reading after the end of the current string.
just like table attributes, field and index attributes must be parsed using the underlying engine, not ha_partition.
Added ADAPTIVE_HASH_INDEX=YES|NO table and index option to InnoDB.
The table and index options only have an effect if InnoDB adaptive hash
index feature is enabled.
- Having the ADAPTIVE_HASH_INDEX TABLE option set to NO will disable
adaptive hash index for all indexes in the table that does not have
the index option adaptive_hash_index=yes.
- Having the ADAPTIVE_HASH_INDEX TABLE option set to YES will enable the
adaptive hash index for all indexes in the table that does not have
the index option adaptive_hash_index=no.
- Using adaptive_hash_index=default deletes the old setting.
- One can also use OFF/ON as the options. This is to make it work similar
as other existing options.
- innodb.adaptive_hash_index has been changed from a bool to an enum with
values OFF, ON and IF_SPECIFIED. If IF_SPECIFIED is used, adaptive
hash index are only used for tables and indexes that specifies
adaptive_hash_index=on.
- The following new options can be used for further optimize adaptive hash
index for an index:
- complete_fields (default 0):
- 0 to the number of columns the key is defined on
- bytes_from_incomplete_fields (default 0):
- This is only usable for memcmp() comparable index fields, such as
VARBINARY or INT. For example, a 3-byte prefix on an INT will
return an identical hash value for 0‥255, another one for 256‥511,
and so on.
- for_equal_hash_point_to_last_record (default 0)
- Default is the first record, known as left_side in the code.
Example: we have an INT column with the values 1,4,10 and bytes=3,
will that hash value point to the record 1 or the record 10?
Note: all values will necessarily have the same hash value
computed on the big endian byte prefix 0x800000, for all of the
values 0x80000001, 0x80000004, 0x8000000a. InnoDB inverts the
sign bit in order to have memcmp() compatible comparison
Example:
CREATE TABLE t1 (a int primary key, b varchar(100), c int,
index (b) adaptive_hash_index=no, index (c))
engine=innodb, adaptive_hash_index=yes;
Notable changes in InnoDB
- btr_search.enabled was changed from a bool to a ulong to be
able to handle options OFF, ON as IF_ENABLED. ulong is needed
to compile with MariaDB enum variables.
- To be able to find all instances where btr_search.enabled was used
I changed all code to use btr_search.get_enabled() when accessing
the value and used btr_search.is_enabled(index) to test if AHI is
enabled for the index.
- btr_search.enabled() was changed to always take two parameters,
resize and value of enabled. This was needed as enabled can now
have values 0, 1, and 2.
Visible user changes:
- select @@global.adaptive_hash_index will now return a string instead
of 0 or 1.
Other things (for Marko)
- Check in buf0buff.cc buf_pool_t::resize(). The first call to
btr_search.enable will enver happen as ahi_disabled is always 0
here.
FIXME: Correctly implement the per-index parameters and adjust the test
innodb_ahi_enable(): Apply the adative_hash_index table and index options to the InnoDB table and index. FIXME: The value 2, which the logic makes use of, is never being used.
fix MSAN/ASAN errors due to use after free of `option_struct` in test `parts.partition_special_innodb`. TODO somebody more knowledgeable about partitions shall review this
- make per-table/per-index `adaptive_hash_index` and per-index
`for_equal_hash_point_to_last_record` options have 3 possible values:
DEFAULT (0), YES (1), NO (2)
- renamed `bytes_from_incomplete_fields` to
`bytes_from_incomplete_field`
- per-index `complete_fields` and `bytes_from_incomplete_field` options
now default to `ULONGLONG_MAX` (which means DEFAULT/unset) and
can be set to any legal value including 0
- `complete_fields` in [0, 64]
- `bytes_from_incomplete_field` in [0, 16383]
- store all AHI related options in per-index `dict_index_t::ahi`
bit-packed 32-bit atomic field `ahi_enabled_fixed_mask`
- static assertions and debug assertions ensure that all options fit
into the 32-bit field
- packing details:
- `enabled`, `adaptive_hash_index` (first 2 bits)
- `fields`, `complete_fields` (7 bit)
- `bytes`, `bytes_from_incomplete_field` (14 bits)
- `left`, `~for_equal_hash_point_to_last_record` (1 bit)
- `is_fields_set`, `fields` set flag (1 bit)
- `is_bytes_set`, `bytes` set flag (1 bit)
- `is_left_set`, `left` set flag (last 1 bit)
- 5 bits spare after `is_left_set`
- remove unused per-table `ahi_enabled` option in `dict_table_t`
- in `innodb_ahi_enable` set per-index options in any case to avoid
stale values being picked up later
- in `innodb_ahi_enable` ensure that the primary key is not updated
twice if both per-table and per-index options are set
- in `btr_sea::resize` avoid losing the previous `btr_sea::enabled`
setting
- in `btr_search_update_hash_ref` apply the per-index AHI options
using bit-masking to override internal heuristic values with user
preferences
- in `innodb.index_ahi_option` replace `ANALYZE FORMAT=JSON` on
`SELECT` over warmed-up AHI with a stored procedure which
checks if AHI is used during a burst of index lookups checking
delta in `adaptive_hash_searches` InnoDB monitor variable as this
is more stable
- in `innodb.index_ahi_option` test a combination of per-table
and per-index AHI options
- in `innodb.index_ahi_option` test that the maximum number of fields
per (secondary) index is 64 (32+32)
- in `innodb.index_ahi_option_debug` test debug builds with
`index_ahi_option_debug_check` debug variable enabled to verify that
the proper per-index AHI options are applied during index lookups
- in `innodb.index_ahi_option_debug` test that illegal per-index AHI
are non-destructive and just lead to no AHI usage
- in `sys_vars.innodb_adaptive_hash_index_basic` replace:
- `for_equal_hash_point_to_last_record=1`
with
- `for_equal_hash_point_to_last_record=no`
to reflect the new 3-value logic
- in `sys_vars.innodb_adaptive_hash_index_basic check that both
`complete_fields` and `bytes_from_incomplete_field` can be set to 0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This is based on work by @montywi. Some InnoDB changes will be needed, as noted in MDEV-37070.
Release Notes
TBD
How can this PR be tested?
Basing the PR against the correct MariaDB version
mainbranch.PR quality check