Fixing integ tests due to missing opensearch-job-scheduler plugin and jayway lib

[yizheliu-amazon]

Description

Fixing integ tests due to missing opensearch-job-scheduler plugin.

This is needed since ML commons introduces new dependency on job-scheduler: https://github.com/opensearch-project/ml-commons/pull/3421/files#diff-520f178549fabd7e74f5b1fedb7275a1434a2ac153fc49e0212455ceeef38e37R47

Before this change, integ test will fail due to below job-scheduler missing error:

| Exception in thread "main" java.lang.IllegalArgumentException: Missing plugin [opensearch-job-scheduler], dependency of [opensearch-ml]
|       at org.opensearch.plugins.PluginsService.addSortedBundle(PluginsService.java:527)
|       at org.opensearch.plugins.PluginsService.sortBundles(PluginsService.java:495)
|       at org.opensearch.plugins.InstallPluginCommand.jarHellCheck(InstallPluginCommand.java:833)
|       at org.opensearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:807)
|       at org.opensearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:852)
|       at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:274)
|       at org.opensearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:248)
|       at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
|       at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
|       at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
|       at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
|       at org.opensearch.cli.Command.main(Command.java:101)
|       at org.opensearch.plugins.PluginCli.main(PluginCli.java:66)

FAILURE: Build failed with an exception.

Also, integ test fail with the error saying below, which is captured in #1145

  2> REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.neuralsearch.processor.SparseEncodingProcessIT.testSparseEncodingProcessor" -Dtests.seed=C9C47530B30641AB -Dtests.security.manager=false -Dtests.locale=cs -Dtests.timezone=America/Fort_Nelson -Druntime.java=21
  2> org.opensearch.client.ResponseException: method [POST], host [http://[::1]:41111], URI [sparse_encoding_index/_doc?refresh], status line [HTTP/1.1 500 Internal Server Error]
    {"error":{"root_cause":[{"type":"null_pointer_exception","reason":"Cannot invoke \"org.opensearch.ml.common.output.model.ModelTensorOutput.getMlModelOutputs()\" because \"modelTensorOutput\" is null"}],"type":"null_pointer_exception","reason":"Cannot invoke \"org.opensearch.ml.common.output.model.ModelTensorOutput.getMlModelOutputs()\" because \"modelTensorOutput\" is null"},"status":500}

Root cause of this error is that jayway lib is not included as runtime dependency

»       at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
»  Caused by: java.lang.NoClassDefFoundError: com/jayway/jsonpath/PathNotFoundException
»       at org.opensearch.ml.common.output.model.ModelTensor.<init>(ModelTensor.java:235) ~[opensearch-ml-common-2.19.0.0.jar:?]
»       at org.opensearch.ml.common.output.model.ModelTensors.<init>(ModelTensors.java:66) ~[opensearch-ml-common-2.19.0.0.jar:?]
»       at org.opensearch.ml.common.output.model.ModelTensorOutput.<init>(ModelTensorOutput.java:44) ~[opensearch-ml-common-2.19.0.0.jar:?]
»       at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) ~[?:?]
»       ... 21 more
»  Caused by: java.lang.ClassNotFoundException: com.jayway.jsonpath.PathNotFoundException

Related Issues

#1145
ML-commons side: opensearch-project/ml-commons#3464

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[q-andy]

Does this resolve #1145?

[owaiskazi19]

LGTM with a minor comment

[weijia-aws]

Does this resolve #1145?

No, the tests are still failing. In fact, I don't think this commit will fix the integ tests, instead it fixes the build failure that caused by:

Exception in thread "main" java.lang.IllegalArgumentException: Missing plugin [opensearch-job-scheduler], dependency of [opensearch-ml]

Which happens before running the integ tests

[vibrantvarun]

Please wait for all CI checks to pass

[martin-gaievski]

is this related to some changes in ml-commons?

NeuralQueryEnricherProcessorIT > testNeuralQueryEnricherProcessor_NeuralSparseSearch_E2EFlow FAILED
    org.opensearch.client.ResponseException: method [POST], host [http://[::1]:44529], URI [/neuralsearch-bwc-testneuralqueryenricherprocessor_neuralsparsesearch_e2eflow/_doc/0?refresh=true], status line [HTTP/1.1 500 Internal Server Error]
    {"error":{"root_cause":[{"type":"null_pointer_exception","reason":"Cannot invoke \"org.opensearch.ml.common.output.model.ModelTensorOutput.getMlModelOutputs()\" because \"modelTensorOutput\" is null"}],"type":"null_pointer_exception","reason":"Cannot invoke \"org.opensearch.ml.common.output.model.ModelTensorOutput.getMlModelOutputs()\" because \"modelTensorOutput\" is null"},

[vibrantvarun]

This issue is coming from ML commons

[vibrantvarun]

Do not merge this PR for now

[yizheliu-amazon]

Same issue as identified in #1145

[vibrantvarun]

@yizheliu-amazon attach the relevant issues in the PR description. The bug opened by @WeijiaZhao on neural

[vibrantvarun]

@yizheliu-amazon I think we need to add this in qa/build.gradle as well

[yizheliu-amazon]

Added same change to qa/build.gradle, but still not working. Reproducing on my local now.

[vibrantvarun]

Hi @yizheliu-amazon here you need to do following things

1. Provide following line in qa/build.gradle

   // json-path 2.9.0 depends on slf4j 2.0.11, which conflicts with the version used by OpenSearch core.
    // Excluding slf4j here since json-path is only used for testing, and logging failures in this context are acceptable.
   // OpenSearch core is using slf4j 1.7.36. Therefore, we cannot change the version here.
    testRuntimeOnly('com.jayway.jsonpath:json-path:2.9.0') {
        exclude group: 'org.slf4j', module: 'slf4j-api'
    }

2. Provide following line in build.gradle

   // json-path 2.9.0 depends on slf4j 2.0.11, which conflicts with the version used by OpenSearch core.
    // Excluding slf4j here since json-path is only used for testing, and logging failures in this context are acceptable.
   // OpenSearch core is using slf4j 1.7.36. Therefore, we cannot change the version here.
    runtimeOnly('com.jayway.jsonpath:json-path:2.9.0') {
        exclude group: 'org.slf4j', module: 'slf4j-api'
    }

3. BWC tests of 2.19.0-SNAPSHOT with 3.0.0 will fail because this change is not present in 2.19 branch. Once we merge this PR it will backported to that branch. POC: Test Pr 2 #1149

[yizheliu-amazon]

Incorporated changes of PR #1148 from @vibrantvarun to this PR, so that Sl4j is excluded, and jayway lib can be used correctly.

[vibrantvarun]

Note: BWC tests with branch 2.19.0-SNAPSHOT with 3.0 will continue to fail until these change is backported to 2.x

[vibrantvarun]

Also BWC tests with 2.18.0 with 3.0.0 are passing because older version is picking dependencies in right fashion. Once this PR goes to 2.x then 2.19 will also be start passing.

Need fresh review as it contains some new changes.

[vibrantvarun]

@junqiu-lei / @martin-gaievski Can one you review this PR now? From my side it is LGTM and green

[junqiu-lei]

@yizheliu-amazon Thanks for the fix, LGTM.