Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set column major segment builder to be on by default #12770

Merged
merged 1 commit into from
Apr 2, 2024

Conversation

ege-st
Copy link
Contributor

@ege-st ege-st commented Apr 1, 2024

This enables the faster Column Major segment builder to be on by default. This feature has been in the wild for several months and since it's purpose is to speed up segment building it makes sense to enable it by default.

Effect:

  1. Any new table will be configured to use column major segment building.
  2. I believe any existing table will continue to use row major segment building unless the config is updated manually.

Testing:

  1. Ran locally to verify that tables used column major segment building.
  2. Ran unit tests to make sure they all passed.

@Jackie-Jiang Jackie-Jiang added the Configuration Config changes (addition/deletion/change in behavior) label Apr 1, 2024
@codecov-commenter
Copy link

codecov-commenter commented Apr 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.58%. Comparing base (59551e4) to head (a7836b7).
Report is 191 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #12770      +/-   ##
============================================
- Coverage     61.75%   61.58%   -0.17%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2463      +27     
  Lines        133233   134745    +1512     
  Branches      20636    20822     +186     
============================================
+ Hits          82274    82982     +708     
- Misses        44911    45568     +657     
- Partials       6048     6195     +147     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 61.53% <100.00%> (-0.18%) ⬇️
java-21 61.45% <100.00%> (-0.17%) ⬇️
skip-bytebuffers-false 61.57% <100.00%> (-0.18%) ⬇️
skip-bytebuffers-true 34.43% <100.00%> (+6.70%) ⬆️
temurin 61.58% <100.00%> (-0.17%) ⬇️
unittests 61.58% <100.00%> (-0.17%) ⬇️
unittests1 46.19% <100.00%> (-0.70%) ⬇️
unittests2 27.94% <0.00%> (+0.20%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang merged commit 4abb2d1 into apache:master Apr 2, 2024
18 of 19 checks passed
@siddharthteotia
Copy link
Contributor

To clarify, this is enabling the column major segment builder for the RT / consuming to immutable segment conversion. Correct ?

I am guessing this column major builder is both space efficient and faster right ?

@ege-st
Copy link
Contributor Author

ege-st commented Apr 4, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Configuration Config changes (addition/deletion/change in behavior)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants