Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Fix auto date histogram rounding assertion bug #17175

Merged
merged 3 commits into from
Jan 30, 2025

Conversation

finnegancarroll
Copy link
Contributor

@finnegancarroll finnegancarroll commented Jan 29, 2025

Backport de59264 from #17023.

Additionally update isUTC to be non-abstract. Breaks backward compatibility with 2.x.
Remove unused ZoneId getter from DocValueFormat.java.

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@finnegancarroll
Copy link
Contributor Author

{"run-benchmark-test": "id_5"}

Copy link
Contributor

❌ Gradle check result for 640534d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@finnegancarroll
Copy link
Contributor Author

@finnegancarroll
Copy link
Contributor Author

{"run-benchmark-test": "id_5"}

Copy link
Contributor

✅ Gradle check result for 640534d: SUCCESS

Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.

Project coverage is 71.98%. Comparing base (2187369) to head (740ecab).
Report is 7 commits behind head on 2.x.

Files with missing lines Patch % Lines
.../src/main/java/org/opensearch/common/Rounding.java 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##                2.x   #17175    +/-   ##
==========================================
  Coverage     71.98%   71.98%            
+ Complexity    65987    65981     -6     
==========================================
  Files          5341     5341            
  Lines        307114   307157    +43     
  Branches      44824    44823     -1     
==========================================
+ Hits         221068   221122    +54     
+ Misses        67608    67501   -107     
- Partials      18438    18534    +96     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@finnegancarroll
Copy link
Contributor Author

{"run-benchmark-test": "id_5"}

@finnegancarroll
Copy link
Contributor Author

Closing this for now to investigate performance regression: #17023 (comment)

@rishabh6788
Copy link
Contributor

Local benchmarks don't replicate this, feel free to open and merge this.
Will investigate.

Copy link
Contributor

❌ Gradle check result for 7fc3198: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@finnegancarroll
Copy link
Contributor Author

github-actions bot and others added 3 commits January 30, 2025 11:50
)

* Add comments explanations for auto date histo increaseRoundingIfNeeded.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Add testFilterRewriteWithTZRoundingRangeAssert() to reproduce auto date histo assertion bug per opensearch-project#16932

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Fix opensearch-project#16932. Ensure optimized path can only increase preparedRounding of agg.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Spotless apply

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Fast fail filter rewrite opt in data histo aggs for non UTC timezones

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Remove redundant UTC check from getInterval().

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Save a call to prepareRounding if roundingIdx is unchanged.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Spotless apply

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Changelog

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Add ZoneId getter for date histo filter rewrite canOptimize check.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Spotless apply

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Disable ff optimzation for composite agg in canOptimize.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Spotless apply

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Handle utc timezone check

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Remove redundant timeZone getter.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Simplify ff prepared rounding check.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

---------

Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
(cherry picked from commit de59264)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Finn Carroll <carrofin@amazon.com>
Copy link
Contributor

✅ Gradle check result for 740ecab: SUCCESS

@jainankitk jainankitk merged commit a79c6e8 into opensearch-project:2.x Jan 30, 2025
48 of 49 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 30, 2025
* Fix auto date histogram rounding assertion bug (#17023)

* Add comments explanations for auto date histo increaseRoundingIfNeeded.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Add testFilterRewriteWithTZRoundingRangeAssert() to reproduce auto date histo assertion bug per #16932

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Fix #16932. Ensure optimized path can only increase preparedRounding of agg.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Spotless apply

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Fast fail filter rewrite opt in data histo aggs for non UTC timezones

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Remove redundant UTC check from getInterval().

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Save a call to prepareRounding if roundingIdx is unchanged.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Spotless apply

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Changelog

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Add ZoneId getter for date histo filter rewrite canOptimize check.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Spotless apply

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Disable ff optimzation for composite agg in canOptimize.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Spotless apply

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Handle utc timezone check

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Remove redundant timeZone getter.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Simplify ff prepared rounding check.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

---------

Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
(cherry picked from commit de59264)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Remove breaking abstract isUTC() getter from Rounding.java.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

* Remove unused ZoneId getter.

Signed-off-by: Finn Carroll <carrofin@amazon.com>

---------

Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
(cherry picked from commit a79c6e8)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
andrross pushed a commit that referenced this pull request Jan 31, 2025
#17211)

* Fix auto date histogram rounding assertion bug (#17023)

* Add comments explanations for auto date histo increaseRoundingIfNeeded.



* Add testFilterRewriteWithTZRoundingRangeAssert() to reproduce auto date histo assertion bug per #16932



* Fix #16932. Ensure optimized path can only increase preparedRounding of agg.



* Spotless apply



* Fast fail filter rewrite opt in data histo aggs for non UTC timezones



* Remove redundant UTC check from getInterval().



* Save a call to prepareRounding if roundingIdx is unchanged.



* Spotless apply



* Changelog



* Add ZoneId getter for date histo filter rewrite canOptimize check.



* Spotless apply



* Disable ff optimzation for composite agg in canOptimize.



* Spotless apply



* Handle utc timezone check



* Remove redundant timeZone getter.



* Simplify ff prepared rounding check.



---------




(cherry picked from commit de59264)


* Remove breaking abstract isUTC() getter from Rounding.java.



* Remove unused ZoneId getter.



---------






(cherry picked from commit a79c6e8)

Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants