Skip to content

Configurable Hash Algorithm and Parameters for Consistent Hash Parent Selection #12567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Configurable Hash Algorithm and Parameters for Consistent Hash Parent Selection #12567

wants to merge 2 commits into from
+842 −186

Conversation

PSUdaemon
Copy link
Contributor

@PSUdaemon PSUdaemon commented Oct 15, 2025 •
edited
Loading

Overview

Makes consistent hash parent selection fully configurable by adding support for:

  • Selectable hash algorithms (SipHash-2-4 and SipHash-1-3)
  • Configurable hash seeds for cryptographic key customization
  • Configurable replica count for hash ring distribution tuning

Configuration is available at three levels: global defaults (records.yaml), per-rule (parent.config), and per-strategy (strategies.yaml).

Motivation

The current implementation hard-codes SipHash-2-4 with zero seeds and 1024 replicas. This change provides flexibility for operators to:

  1. Choose faster hash algorithms based on performance requirements
  2. Customize hash seeds for security or cache isolation
  3. Tune replica counts for optimal distribution vs memory usage
  4. Configure these parameters globally or per-rule/strategy

Configuration

Parameters

Parameter Type Default Description
hash_algorithm string siphash24 Hash algorithm: siphash24 or siphash13
hash_seed0 uint64_t 0 First 64 bits of hash seed (decimal integer)
hash_seed1 uint64_t 0 Second 64 bits of hash seed (decimal integer)
hash_replicas int 1024 Virtual nodes per host on hash ring

Global Defaults (records.yaml)

http:
  parent_proxy:
    consistent_hash_algorithm: siphash24
    consistent_hash_seed0: 0
    consistent_hash_seed1: 0
    consistent_hash_replicas: 1024

Per-Rule Configuration (parent.config)

  dest_domain=example.com parent=p1:80,p2:80 round_robin=consistent_hash hash_algorithm=siphash13 hash_seed0=12345678901234567 hash_replicas=2048

Per-Strategy Configuration (strategies.yaml)

strategies:
  - strategy: 'my-strategy'
    policy: consistent_hash
    hash_algorithm: siphash13
    hash_seed0: 12345678901234567
    hash_seed1: 9876543210987654321
    hash_replicas: 2048
    groups:
      - *primary_group

Hash Algorithms

  • SipHash-2-4 (siphash24, default): Cryptographically strong, DoS-resistant
  • SipHash-1-3 (siphash13): ~50% faster, still DoS-resistant

Hash Seeds:

  • SipHash uses both hash_seed0 and hash_seed1 as a 128-bit key
  • Future 64-bit algorithms (XXH3, RapidHash) will use only hash_seed0

Migration Considerations

Warning: Changing hash_algorithm, hash_seed0, or hash_seed1 will redistribute requests across parent proxies, causing:

  • Cache churn
  • Increased origin load during transition
  • Temporary cache hit rate reduction

Plan changes during low-traffic periods and monitor origin load.

@PSUdaemon PSUdaemon requested a review from zwoop October 15, 2025 22:43
@PSUdaemon PSUdaemon self-assigned this Oct 15, 2025
@PSUdaemon PSUdaemon force-pushed the parent_selection_hash_extension branch 5 times, most recently from 235ada1 to 5edaa90 Compare October 16, 2025 00:00
@jrushford
Copy link
Contributor

jrushford commented Oct 16, 2025 via email

@PSUdaemon PSUdaemon force-pushed the parent_selection_hash_extension branch 4 times, most recently from 056624f to fbdaa1a Compare October 16, 2025 01:23
@PSUdaemon
Copy link
Contributor Author

@PSUdaemon this is relevant to strategies also as it uses the SipHash-2-4 for its consistent hash used for parent selection. You might consider a PR for strategies.yaml. I believe that strategies will deprecate parent selection in the future. Regards Jrushford

@jrushford, I looked into this and seems like a logical next PR. If the community likes this one and merges it, I'll do that next.

@PSUdaemon PSUdaemon force-pushed the parent_selection_hash_extension branch from fbdaa1a to f27095f Compare October 16, 2025 01:55
@PSUdaemon PSUdaemon requested a review from jrushford October 16, 2025 02:30
@PSUdaemon PSUdaemon force-pushed the parent_selection_hash_extension branch from e2591b4 to db539a1 Compare October 16, 2025 18:38
@maskit
Copy link
Member

maskit commented Oct 17, 2025

It seems like the description was generated by AI, which is fine and sorry if it wasn't. Was the code generated by AI? If so, can you make sure that the code is compliant with the guideline from ASF?
https://www.apache.org/legal/generative-tooling.html

@bryancall bryancall added this to the 10.2.0 milestone Oct 20, 2025
@bryancall bryancall requested a review from traeak October 20, 2025 22:06
@PSUdaemon PSUdaemon force-pushed the parent_selection_hash_extension branch from db539a1 to b14649b Compare October 21, 2025 23:20
@PSUdaemon
Copy link
Contributor Author

It seems like the description was generated by AI, which is fine and sorry if it wasn't. Was the code generated by AI? If so, can you make sure that the code is compliant with the guideline from ASF? https://www.apache.org/legal/generative-tooling.html

I'm not sure if I totally follow this, but it appears to be more about copyright. I removed WyHash to remove questions about the source of code. We can add that or other hash functions later and debate that issue there. This commit also now has more infrastructure that Craig and @jrushford requested.

I can attest that the other code is safe to include. The SipHash implementation is just a refactor of the existing version and the rest is adding configs and uses the local code style.

@PSUdaemon PSUdaemon force-pushed the parent_selection_hash_extension branch from b14649b to 0b777e4 Compare October 21, 2025 23:25
@PSUdaemon PSUdaemon changed the title Configurable Hash Algorithm for Consistent Hash Parent Selection (also add SipHash-1-3 and Wyhash v4.1) Make Consistent Hash Ring more configurable Oct 21, 2025
@PSUdaemon PSUdaemon changed the title Make Consistent Hash Ring more configurable Configurable Hash Algorithm and Parameters for Consistent Hash Parent Selection Oct 21, 2025
maskit
maskit reviewed Oct 23, 2025
View reviewed changes
src/proxy/http/remap/NextHopConsistentHash.cc
uint64_t hash_key = 0;
// Create hash instance based on configured algorithm and seeds
std::unique_ptr<ATSHash64> hash;
if (hash_algorithm == "siphash13") {
Copy link
Member

@maskit maskit Oct 23, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it's almost a duplicate of ParentConsistentHash::createHashInstance. Can we have just one place to do this?

@maskit
Copy link
Member

maskit commented Oct 23, 2025

I'm not sure if I totally follow this, but it appears to be more about copyright. I removed WyHash to remove questions about the source of code. We can add that or other hash functions later and debate that issue there. This commit also now has more infrastructure that Craig and @jrushford requested.

Thank you for the update.

This is just a suggestion, but it would be more interesting if we could add the support for WyHash or other algorithms without changing ATS core. Apparently, WyHash has a successor (Rapidhash). Adding the support for WyHash might only increase the size of code we have to maintain. I'd suggest introducing TSHashAlgoRegister(const char *name, HashAlgoMethod *method) or some such so that users can add preferred hash algorithms by using/writing TS plugins. It'd also allow us to write a "hash" algorithm that is more like mapping and that could be useful for testing or experiments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maskit maskit maskit left review comments

@zwoop zwoop Awaiting requested review from zwoop

@jrushford jrushford Awaiting requested review from jrushford

@traeak traeak Awaiting requested review from traeak

At least 1 approving review is required to merge this pull request.

Assignees

@PSUdaemon PSUdaemon

Labels

Parent Proxy

Projects

None yet

Milestone

10.2.0

Development

Successfully merging this pull request may close these issues.

4 participants

@PSUdaemon @jrushford @maskit @bryancall