Skip to content

Conversation

@SkyFan2002
Copy link
Member

@SkyFan2002 SkyFan2002 commented Dec 3, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

build the runtime filter during the HashJoin block collection process, so that this part of the computation can be naturally parallelized.

before:
image
after:

image

Refactored the concurrent Bloom filter construction: now it is built concurrently using atomic variables, replacing the previous approach where each thread built its own filter and the results were merged afterward.

╭────────────────────────────────────────────────────────────────────────────────────────────╮
│ query_id │     v852_time_s     │     pr_s     │       diff_s      │   ratio_percent   │
│  String  │  Nullable(Float64) │  Nullable(Float64) │ Nullable(Float64) │ Nullable(Float64) │
├──────────┼────────────────────┼────────────────────┼───────────────────┼───────────────────┤
│ q21      │  63.46466573079427 │  54.11199951171875 │             -9.35 │            -14.74 │
│ q9       │  65.49333318074544 │  62.13600158691406 │             -3.36 │             -5.13 │
│ q3       │  43.02449989318848 │ 41.281999588012695 │             -1.74 │             -4.05 │
│ q10      │ 30.885334014892578 │  29.36050033569336 │             -1.52 │             -4.94 │
│ q14      │ 15.077999750773111 │ 14.332499980926514 │             -0.75 │             -4.94 │
│ q13      │ 21.867666244506836 │  21.32599925994873 │             -0.54 │             -2.48 │
│ q22      │ 7.2423332532246905 │  6.700000047683716 │             -0.54 │             -7.49 │
│ q17      │  12.84766674041748 │ 12.319499969482422 │             -0.53 │             -4.11 │
│ q18      │  28.43233362833659 │  28.05049991607666 │             -0.38 │             -1.34 │
│ q4       │ 31.922666549682617 │ 31.586999893188477 │             -0.34 │             -1.05 │
│ q7       │  22.17566680908203 │ 21.889500617980957 │             -0.29 │             -1.29 │
│ q19      │  14.81499989827474 │ 14.558000087738037 │             -0.26 │             -1.73 │
│ q1       │  11.45466677347819 │  11.21150016784668 │             -0.24 │             -2.12 │
│ q12      │  13.55233351389567 │ 13.398000240325928 │             -0.15 │             -1.14 │
│ q20      │  7.464000066121419 │  7.398499965667725 │             -0.07 │             -0.88 │
│ q2       │  5.857666651407878 │  5.829499959945679 │             -0.03 │             -0.48 │
│ q11      │  2.699666738510132 │  2.694499969482422 │             -0.01 │             -0.19 │
│ q6       │  8.589999834696451 │  8.609500408172607 │              0.02 │              0.23 │
│ q16      │   2.98799999554952 │  3.097499966621399 │              0.11 │              3.66 │
│ q8       │ 22.851999918619793 │  23.10550022125244 │              0.25 │              1.11 │
│ q5       │   24.3559996287028 │ 24.728999137878418 │              0.37 │              1.53 │
│ q15      │ 10.366666793823242 │  10.75950002670288 │              0.39 │              3.79 │
╰────────────────────────────────────────────────────────────────────────────────────────────╯
22 rows read in 2.066 sec. Processed 84 rows, 13.16 KiB (40.66 rows/s, 6.37 KiB/s)

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-refactor this PR changes the code base without new features or bugfix label Dec 3, 2025
@SkyFan2002 SkyFan2002 added the ci-cloud Build docker image for cloud test label Dec 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 3, 2025

Docker Image for PR

  • tag: pr-19058-cec340e-1764784971

note: this image tag is only available for internal use.

@SkyFan2002 SkyFan2002 marked this pull request as ready for review December 4, 2025 07:20
@SkyFan2002 SkyFan2002 requested a review from zhang2014 December 4, 2025 07:20
@zhang2014 zhang2014 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Dec 4, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2025

Docker Image for PR

  • tag: pr-19058-cf70ad4-1764840657

note: this image tag is only available for internal use.

@SkyFan2002 SkyFan2002 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Dec 8, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 8, 2025

Docker Image for PR

  • tag: pr-19058-80ee987-1765188163

note: this image tag is only available for internal use.

SkyFan2002 and others added 3 commits December 8, 2025 20:49
Co-authored-by: Winter Zhang <coswde@gmail.com>
@SkyFan2002 SkyFan2002 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Dec 8, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 8, 2025

Docker Image for PR

  • tag: pr-19058-3999808-1765204696

note: this image tag is only available for internal use.

@zhang2014 zhang2014 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Dec 9, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 9, 2025

Docker Image for PR

  • tag: pr-19058-2d0d8a2-1765266827

note: this image tag is only available for internal use.

@zhang2014 zhang2014 merged commit 6e63033 into databendlabs:main Dec 9, 2025
100 of 102 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-refactor this PR changes the code base without new features or bugfix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants