refactor: reduce the memory of the HashIndex in the AggregateHashTable #19046

forsaken628 · 2025-12-02T08:22:17Z

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Introduce RowID, using smaller types to represent rows
No longer need temp_vector for compare
Aggregate supports const columns
Refine Aggregate unit test

Tests

Unit Test
Logic Test
Benchmark Test
No Test - Explain why

Type of change

Bug Fix (non-breaking change which fixes an issue)
New Feature (non-breaking change which adds functionality)
Breaking Change (fix or feature that could cause existing functionality not to work as expected)
Documentation Update
Refactoring
Performance Improvement
Other (please describe):

This change is

github-actions · 2025-12-02T13:47:13Z

Docker Image for PR

tag: pr-19046-485a16d-1764683080

note: this image tag is only available for internal use.

github-actions · 2025-12-02T14:15:28Z

ClickBench Report

This reverts commit 1e27e0f.

github-actions · 2025-12-02T18:32:43Z

Docker Image for PR

tag: pr-19046-ab10226-1764700210

note: this image tag is only available for internal use.

github-actions · 2025-12-02T19:00:24Z

ClickBench Report

github-actions · 2025-12-03T10:37:34Z

Docker Image for PR

tag: pr-19046-39fb5b9-1764758008

note: this image tag is only available for internal use.

github-actions · 2025-12-03T13:18:28Z

Docker Image for PR

tag: pr-19046-6697d22-1764767755

note: this image tag is only available for internal use.

github-actions · 2025-12-03T15:04:59Z

Docker Image for PR

tag: pr-19046-21c318b-1764774146

note: this image tag is only available for internal use.

github-actions · 2025-12-03T23:17:05Z

Docker Image for PR

tag: pr-19046-b6a4e76-1764803676

note: this image tag is only available for internal use.

github-actions · 2025-12-04T03:49:43Z

Docker Image for PR

tag: pr-19046-d0ad800-1764820048

note: this image tag is only available for internal use.

everpcpc · 2025-12-04T05:15:01Z

https://benchmark.databend.com/clickbench/pr/19046/19913624838/hits.html

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

src/query/expression/src/aggregate/row_ptr.rs

BohuTANG · 2025-12-09T13:39:19Z

@codex review

chatgpt-codex-connector · 2025-12-09T13:39:26Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

github-actions · 2025-12-09T14:41:16Z

Docker Image for PR

tag: pr-19046-ca96af3-1765291122

note: this image tag is only available for internal use.

github-actions · 2025-12-09T15:33:04Z

ClickBench Report

hits: https://benchmark.databend.com/clickbench/pr/19046/20065492590/hits.html
tpch100: https://benchmark.databend.com/clickbench/pr/19046/20065492590/tpch100.html
tpch1000: https://benchmark.databend.com/clickbench/pr/19046/20065492590/tpch1000.html

github-actions · 2025-12-10T14:05:53Z

Docker Image for PR

tag: pr-19046-235e4ff-1765375400

note: this image tag is only available for internal use.

github-actions · 2025-12-10T14:57:06Z

ClickBench Report

hits: https://benchmark.databend.com/clickbench/pr/19046/20099687402/hits.html
tpch100: https://benchmark.databend.com/clickbench/pr/19046/20099687402/tpch100.html
tpch1000: https://benchmark.databend.com/clickbench/pr/19046/20099687402/tpch1000.html

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2025-12-11T01:19:21Z

src/query/functions/src/aggregates/adaptors/aggregate_combinator_if.rs

+            BlockEntry::Const(Scalar::Boolean(v), _, _) => {
+                if *v {
+                    return Ok(());
+                }
+                self.nested.accumulate_keys(


Constant IF predicate drops rows

In accumulate_keys the constant predicate branch is inverted: when the filter argument is BlockEntry::Const(Scalar::Boolean(true), …) the function returns immediately and skips accumulating any rows, while the false case falls through to the nested aggregate (lines below). For aggregates using the _if combinator, a constant TRUE filter should include all rows and a constant FALSE should skip them; current logic flips that behavior, producing zeroed aggregates for always-true filters and counting rows when the filter is always false.

Useful? React with 👍 / 👎.

reduce memory

f7971fc

github-actions bot added the pr-refactor this PR changes the code base without new features or bugfix label Dec 2, 2025

hash match

1e27e0f

forsaken628 added the ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits label Dec 2, 2025

Revert "hash match"

ad5a4a8

This reverts commit 1e27e0f.

forsaken628 added ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits and removed ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits labels Dec 2, 2025

forsaken628 added 2 commits December 3, 2025 17:03

x

b6e1fef

x

963cd88

forsaken628 added ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits and removed ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits labels Dec 3, 2025

everpcpc added ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits and removed ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits labels Dec 3, 2025

everpcpc added ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits and removed ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits labels Dec 4, 2025

everpcpc removed the ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits label Dec 4, 2025

forsaken628 marked this pull request as ready for review December 8, 2025 05:48

forsaken628 requested a review from sundy-li December 8, 2025 05:50

chatgpt-codex-connector bot reviewed Dec 8, 2025

View reviewed changes

src/query/expression/src/aggregate/row_ptr.rs Show resolved Hide resolved

const column

6a53537

sundy-li approved these changes Dec 9, 2025

View reviewed changes

forsaken628 added 7 commits December 9, 2025 11:35

test

5a178fc

fix

7ba4b45

Merge remote-tracking branch 'up/main' into aggr_table

5532745

fix

2d63b19

combinator if

dc5f5d1

sum zero

083b678

test

734a73f

BohuTANG added ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits and removed ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits labels Dec 9, 2025

forsaken628 marked this pull request as draft December 9, 2025 23:21

forsaken628 added 5 commits December 10, 2025 08:55

count

1839fd2

bitmap

1d45db3

test

205535e

update

1f289ea

nullable

6f37419

forsaken628 added ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits and removed ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits labels Dec 10, 2025

forsaken628 marked this pull request as ready for review December 11, 2025 01:09

chatgpt-codex-connector bot reviewed Dec 11, 2025

View reviewed changes

refactor: reduce the memory of the HashIndex in the AggregateHashTable #19046

Are you sure you want to change the base?

refactor: reduce the memory of the HashIndex in the AggregateHashTable #19046

Conversation

forsaken628 commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests

Type of change

Uh oh!

github-actions bot commented Dec 2, 2025

Docker Image for PR

Uh oh!

github-actions bot commented Dec 2, 2025

ClickBench Report

Uh oh!

github-actions bot commented Dec 2, 2025

Docker Image for PR

Uh oh!

github-actions bot commented Dec 2, 2025

ClickBench Report

Uh oh!

github-actions bot commented Dec 3, 2025

Docker Image for PR

Uh oh!

github-actions bot commented Dec 3, 2025

Docker Image for PR

Uh oh!

github-actions bot commented Dec 3, 2025

Docker Image for PR

Uh oh!

github-actions bot commented Dec 3, 2025

Docker Image for PR

Uh oh!

github-actions bot commented Dec 4, 2025

Docker Image for PR

Uh oh!

everpcpc commented Dec 4, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

BohuTANG commented Dec 9, 2025

Uh oh!

chatgpt-codex-connector bot commented Dec 9, 2025

Uh oh!

github-actions bot commented Dec 9, 2025

Docker Image for PR

Uh oh!

github-actions bot commented Dec 9, 2025

ClickBench Report

Uh oh!

github-actions bot commented Dec 10, 2025

Docker Image for PR

Uh oh!

github-actions bot commented Dec 10, 2025

ClickBench Report

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Dec 11, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

forsaken628 commented Dec 2, 2025 •

edited

Loading