[JPMS Support] Initial phase-0 Refactor of org.apache.lucene codebase to eliminate top level split packages

[prudhvigodithi]

Description

• Refactored most of the code part of :server org.apache.lucene.

• Removed server/src/main/java/org/apache/lucene/util/packed which is used for CuckooFilter. Moved the implementation directly to CuckooFilter class as Packed64.

• Removed server/src/main/java/org/apache/lucene/index/OneMergeHelper.java (used by OpenSearchConcurrentMergeScheduler), moved some its methods to OpenSearchConcurrentMergeScheduler (reuired few logging changes).

• From https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/index/SegmentInfo.java#L98, updated the ShuffleForcedMergePolicy to use addDiagnostics.

• Updated the server/src/main/java/org/opensearch/index/search/QueryStringQueryParser.java to to the more standard QueryParser (from XQueryParser) as the base class. Looks like we dont need handleBareFuzzy anymore.

• Updated CustomFieldHighlighter.java with getFieldOffsetStrategy method.

• To server/src/main/java/org/opensearch/lucene/util/automaton/MinimizationOperations.java added new Automaton class copied from https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/util/automaton/Operations.java . Also removed transition.transitionUpto = -1; as from lucene https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/util/automaton/Transition.java#L51 transitionUpto is already initialized to -1

• Pending org.apache.lucene.codecs and org.apache.lucene.index are part of Star Tree implementation.

• The pending org.apache.lucene.search.grouping I will address this in upcoming PR's.

Related Issues

Part of #8110

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 5c0e270: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 4788400: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 4788400: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[reta]

Thanks @prudhvigodithi , the general direction definitely makes sense to me, but some changes may lead to regressions or functional changes, that we should try to avoid at all costs (I think), thank you.

[prudhvigodithi]

Thanks @reta, I see you pointed one #17241 (comment), I will double check but is there anything else we should avoid ?
Adding @msfroh @andrross @getsaurabh02

[reta]

Thanks @reta, I see you pointed one #17241 (comment), I will double check but is there anything else we should avoid ?

I would be surprised to see no test failures ...

[github-actions]

✅ Gradle check result for b3347c3: SUCCESS

[codecov]

Codecov Report

Attention: Patch coverage is 86.66667% with 14 lines in your changes missing coverage. Please review.

Project coverage is 72.45%. Comparing base (faabd10) to head (17a2020).
Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
.../java/org/opensearch/common/util/CuckooFilter.java	82.43%	1 Missing and 12 partials ⚠️
.../lucene/util/automaton/MinimizationOperations.java	95.83%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #17241      +/-   ##
============================================
+ Coverage     72.36%   72.45%   +0.09%     
+ Complexity    65733    65662      -71     
============================================
  Files          5318     5314       -4     
  Lines        305674   305141     -533     
  Branches      44349    44265      -84     
============================================
- Hits         221197   221092     -105     
+ Misses        66341    65945     -396     
+ Partials      18136    18104      -32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

❕ Gradle check result for 4fc0267: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

❌ Gradle check result for 17a2020: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 17a2020: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 17a2020: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 17a2020: SUCCESS

[prudhvigodithi]

Got green for gradle check @msfroh @reta please see if we can get this merged.
Thanks