Skip to content

harden tests for heap graph reconstruction #543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

harden tests for heap graph reconstruction #543

wants to merge 10 commits into from

Conversation

@sam-herman
Copy link
Contributor

@sam-herman sam-herman commented Oct 15, 2025 •
edited
Loading

Description

  • Harden test for the cases where heap graph reconstruction and ravv to graph ordinal mappings are not the identity mapping.
  • Fix bug when graph ord mapping is used with the remmaped RAVV
  • Move to remmaped interface for buildAndMerge to avoid bugs of remmpaing ords twice
  • Add interface for providing the executor pools for the buildAndMerge method -important for downstream dependencies with centralized resource management

@github-actions
Copy link
Contributor

github-actions bot commented Oct 15, 2025

Before you submit for review:

  • Does your PR follow guidelines from CONTRIBUTIONS.md?
  • Did you summarize what this PR does clearly and concisely?
  • Did you include performance data for changes which may be performance impacting?
  • Did you include useful docs for any user-facing changes or features?
  • Did you include useful javadocs for developer oriented changes, explaining new concepts or key changes?
  • Did you trigger and review regression testing results against the base branch via Run Bench Main?
  • Did you adhere to the code formatting guidelines (TBD)
  • Did you group your changes for easy review, providing meaningful descriptions for each commit?
  • Did you ensure that all files contain the correct copyright header?

If you did not complete any of these, then please explain below.

@sam-herman sam-herman force-pushed the harden-tests-for-heap-graph-reconstruction branch from e3fd3be to ae4120c Compare October 29, 2025 02:24
@sam-herman
harden tests for heap graph reconstruction
98507a5 
fix minor bug in construction

Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman sam-herman force-pushed the harden-tests-for-heap-graph-reconstruction branch from ae4120c to 98507a5 Compare October 29, 2025 02:35
@sam-herman
fix bug with incremental reconstruction
c3c2820 
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman
reduce sensitivity to make tests less flakey
ff336e7 
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman
switch tests to remmaped interface
0ab3afe 
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman
add tests for non identity incremental construction
78eca89 
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman
fix javadoc issue
babcc2d 
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@marianotepper
Copy link
Contributor

marianotepper commented Oct 29, 2025

There seems to be a failure from one of the tests that this PR is adding. In testIncrementalInsertionFromOnDiskIndex_withNonIdentityOrdinalMapping

@sam-herman
add additional queries to be more statistically significant for recal…
a46fa4d 
…l calculation

Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman
add interface that allows to provide executor pools
a773308 
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman
remove left over comment
c273c19 
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman
Copy link
Contributor Author

sam-herman commented Oct 29, 2025

There seems to be a failure from one of the tests that this PR is adding. In testIncrementalInsertionFromOnDiskIndex_withNonIdentityOrdinalMapping

Yeah, it is now more robust, the recall calculation need to be more statistically significant so I increased number of vectors + beamsize and number of query vectors participating in average recall calculation.

marianotepper
marianotepper reviewed Oct 29, 2025
View reviewed changes
Copy link
Contributor

@marianotepper marianotepper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments here and there.

jvector-base/src/main/java/io/github/jbellis/jvector/graph/GraphIndexBuilder.java Outdated
return reallyBuild(ravv);
}

public ImmutableGraphIndex build(RandomAccessVectorValues ravv, int[] graphToRavvOrdMap) {
Copy link
Contributor

@marianotepper marianotepper Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not add this method here. It should be the responsibility of the caller to use the remapped RAVV, not a conversion that we want done by JV. Just like the change in buildAndMergeNewNodes

Copy link
Contributor Author

@sam-herman sam-herman Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, will remove it!

@sam-herman
PR feedback
b66349b 
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@sam-herman
Copy link
Contributor Author

sam-herman commented Oct 29, 2025

One of the tests failed due to a different test not related to this change:

Error: 1,882 [ERROR] io.github.jbellis.jvector.quantization.TestCompressedVectors
Error: 1,882 [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:643)
Error: 1,882 [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.lambda$null$3(ForkStarter.java:350)
Error: 1,882 [ERROR] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
Error: 1,882 [ERROR] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
Error: 1,882 [ERROR] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
Error: 1,882 [ERROR] 	at java.base/java.lang.Thread.run(Thread.java:1570)
Error: 1,882 [ERROR] -> [Help 1]

I am re-running this test so we can have a whole green panel

@marianotepper marianotepper self-requested a review October 29, 2025 17:18
marianotepper
marianotepper approved these changes Oct 29, 2025
View reviewed changes
Copy link
Contributor

@marianotepper marianotepper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@marianotepper marianotepper marianotepper approved these changes

@MarkWolters MarkWolters Awaiting requested review from MarkWolters MarkWolters is a code owner

@jshook jshook Awaiting requested review from jshook jshook is a code owner

@tlwillke tlwillke Awaiting requested review from tlwillke tlwillke is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sam-herman @marianotepper