Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--jvm tries to find Java executable system-wide. #11500

Open
wants to merge 27 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ca2be99
--jvm tries to find Java executable system-wide.
Akirathan Nov 6, 2024
19b0c3d
Remove outdated GraalVMComponentUpdater and related functionality fro…
Akirathan Nov 7, 2024
b91121f
runtime-version-manager does not depend on akka-http
Akirathan Nov 7, 2024
d2e8994
Extract GraalVersionManager from RuntimeVersionManager
Akirathan Nov 7, 2024
b38928b
engine-runner depends on runtime-version-manager
Akirathan Nov 7, 2024
b9792ab
JavaFinder uses GraalVersionManager to find the java executable
Akirathan Nov 7, 2024
6b54fcc
Generate javaVersion into version-output
Akirathan Nov 7, 2024
7149cf3
GraalVMVersion is comparable
Akirathan Nov 7, 2024
d6bf5a7
Add some tests to SemVer parsing
Akirathan Nov 7, 2024
1455b02
JavaFinder tries to find newer runtime than the one used for building
Akirathan Nov 7, 2024
586fc73
Fix version-output versions mismatch
Akirathan Nov 7, 2024
dd357ca
Update docs
Akirathan Nov 7, 2024
25d8110
runtimeSearchPath does not have to be a directory
Akirathan Nov 8, 2024
c356a0c
Fix some build failures in tests
Akirathan Nov 8, 2024
ec38e0b
docs
Akirathan Nov 8, 2024
b89db13
Merge branch 'develop' into wip/akirathan/11274-jvm-cmdlineopt-ni
Akirathan Nov 13, 2024
fd52bbf
Make JavaFinder Windows friendly
Akirathan Nov 13, 2024
46022f2
GraalVersionManager is final
Akirathan Nov 13, 2024
ea3af5f
Add test for --jvm arg
Akirathan Nov 13, 2024
df1e2ec
Log found JDK
Akirathan Nov 14, 2024
ed31ffc
Add timeout to waiting for subprocess
Akirathan Nov 14, 2024
9fb23d7
Update licenses - akka-http was removed from launcher
Akirathan Nov 14, 2024
45b7889
When testing --jvm in CI, don't override ENSO_DATA_DIRECTORY env var
Akirathan Nov 14, 2024
f14333d
Run --jvm test only on Linux
Akirathan Nov 14, 2024
184c245
On Windows, java.exe is tried to be found on PATH
Akirathan Nov 14, 2024
fb25d57
Revert "Run --jvm test only on Linux"
Akirathan Nov 14, 2024
049b991
Reapply "Run --jvm test only on Linux"
Akirathan Nov 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 26 additions & 3 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -721,6 +721,7 @@ lazy val componentModulesPaths =
(`runtime-instrument-runtime-server` / Compile / exportedModuleBin).value,
(`runtime-language-arrow` / Compile / exportedModuleBin).value,
(`runtime-language-epb` / Compile / exportedModuleBin).value,
(`runtime-version-manager` / Compile / exportedModuleBin).value,
(`persistance` / Compile / exportedModuleBin).value,
(`cli` / Compile / exportedModuleBin).value,
(`json-rpc-server` / Compile / exportedModuleBin).value,
Expand Down Expand Up @@ -1613,7 +1614,8 @@ lazy val `version-output` = (project in file("lib/scala/version-output"))
defaultDevEnsoVersion = defaultDevEnsoVersion,
ensoVersion = ensoVersion,
scalacVersion = scalacVersion,
graalVersion = graalVersion,
graalVersion = graalMavenPackagesVersion,
javaVersion = graalVersion,
currentEdition = currentEdition
)
}.taskValue
Expand Down Expand Up @@ -3525,6 +3527,7 @@ lazy val `engine-runner` = project
(`pkg` / Compile / exportedModule).value,
(`engine-runner-common` / Compile / exportedModule).value,
(`runtime-parser` / Compile / exportedModule).value,
(`runtime-version-manager` / Compile / exportedModule).value,
(`version-output` / Compile / exportedModule).value,
(`engine-common` / Compile / exportedModule).value,
(`polyglot-api` / Compile / exportedModule).value,
Expand Down Expand Up @@ -3700,6 +3703,7 @@ lazy val `engine-runner` = project
.dependsOn(`distribution-manager`)
.dependsOn(`edition-updater`)
.dependsOn(`runtime-parser`)
.dependsOn(`runtime-version-manager`)
.dependsOn(`logging-service`)
.dependsOn(`logging-service-logback` % Runtime)
.dependsOn(`engine-runner-common`)
Expand Down Expand Up @@ -4337,15 +4341,34 @@ lazy val `connected-lock-manager-server` = project

lazy val `runtime-version-manager` = project
.in(file("lib/scala/runtime-version-manager"))
.enablePlugins(JPMSPlugin)
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
scalaModuleDependencySetting,
mixedJavaScalaProjectSetting,
resolvers += Resolver.bintrayRepo("gn0s1s", "releases"),
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.apache.commons" % "commons-compress" % commonsCompressVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
akkaHttp
"org.scalatest" %% "scalatest" % scalatestVersion % Test
),
Compile / moduleDependencies ++= Seq(
"org.apache.commons" % "commons-compress" % commonsCompressVersion,
"org.slf4j" % "slf4j-api" % slf4jVersion
),
Compile / internalModuleDependencies := Seq(
(`cli` / Compile / exportedModule).value,
Akirathan marked this conversation as resolved.
Show resolved Hide resolved
(`distribution-manager` / Compile / exportedModule).value,
(`downloader` / Compile / exportedModule).value,
Akirathan marked this conversation as resolved.
Show resolved Hide resolved
(`editions` / Compile / exportedModule).value,
(`edition-updater` / Compile / exportedModule).value,
(`logging-utils` / Compile / exportedModule).value,
(`pkg` / Compile / exportedModule).value,
(`semver` / Compile / exportedModule).value,
(`scala-libs-wrapper` / Compile / exportedModule).value,
(`scala-yaml` / Compile / exportedModule).value,
(`version-output` / Compile / exportedModule).value
)
)
.dependsOn(pkg)
Expand Down
21 changes: 20 additions & 1 deletion build/build/src/engine/context.rs
Original file line number Diff line number Diff line change
Expand Up @@ -664,6 +664,7 @@ pub async fn runner_sanity_test(
.bin
.join("enso")
.with_executable_extension();

let test_base = Command::new(&enso)
.args(["--run", repo_root.test.join("Base_Tests").as_str()])
.set_env(ENSO_DATA_DIRECTORY, engine_package)?
Expand All @@ -686,7 +687,25 @@ pub async fn runner_sanity_test(
.run_ok()
.await;

test_base.and(test_internal_base).and(test_geo)
let all_cmds = test_base.and(test_internal_base).and(test_geo);

// The following test does not actually run anything, it just checks if the engine
// can accept `--jvm` argument and evaluates something.
if TARGET_OS != OS::Windows {
Akirathan marked this conversation as resolved.
Show resolved Hide resolved
let test_jvm_arg = Command::new(&enso)
.args([
"--jvm",
"--run",
repo_root.test.join("Base_Tests").as_str(),
"__NON_EXISTING_TEST__",
])
.set_env(ENSO_DATA_DIRECTORY, engine_package)?
.run_ok()
.await;
all_cmds.and(test_jvm_arg)
} else {
all_cmds
}
} else {
Ok(())
}
Expand Down
15 changes: 0 additions & 15 deletions distribution/launcher/THIRD-PARTY/NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -36,21 +36,6 @@ The license file can be found at `licenses/APACHE2.0`.
Copyright notices related to this dependency can be found in the directory `com.typesafe.akka.akka-actor_2.13-2.6.20`.


'akka-http-core_2.13', licensed under the Apache-2.0, is distributed with the launcher.
The license file can be found at `licenses/APACHE2.0`.
Copyright notices related to this dependency can be found in the directory `com.typesafe.akka.akka-http-core_2.13-10.2.10`.


'akka-http_2.13', licensed under the Apache-2.0, is distributed with the launcher.
The license file can be found at `licenses/APACHE2.0`.
Copyright notices related to this dependency can be found in the directory `com.typesafe.akka.akka-http_2.13-10.2.10`.


'akka-parsing_2.13', licensed under the Apache-2.0, is distributed with the launcher.
The license file can be found at `licenses/APACHE2.0`.
Copyright notices related to this dependency can be found in the directory `com.typesafe.akka.akka-parsing_2.13-10.2.10`.


'akka-slf4j_2.13', licensed under the Apache-2.0, is distributed with the launcher.
The license file can be found at `licenses/APACHE2.0`.
Copyright notices related to this dependency can be found in the directory `com.typesafe.akka.akka-slf4j_2.13-2.6.20`.
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,6 @@ object DefaultManagers {
lazy val temporaryDirectoryManager =
TemporaryDirectoryManager(distributionManager, defaultResourceManager)

/** Default [[RuntimeComponentConfiguration]]. */
lazy val componentConfig: RuntimeComponentConfiguration =
new GraalVMComponentConfiguration

/** Creates a [[RuntimeVersionManager]] that uses the default distribution. */
def runtimeVersionManager(
globalCLIOptions: GlobalCLIOptions,
Expand All @@ -55,12 +51,11 @@ object DefaultManagers {
alwaysInstallMissing
),
distributionManager,
new GraalVersionManager(distributionManager, LauncherEnvironment),
temporaryDirectoryManager,
defaultResourceManager,
EngineRepository.defaultEngineReleaseProvider,
GraalCEReleaseProvider.default,
componentConfig,
RuntimeComponentUpdaterFactory.Default,
InstallerKind.Launcher
)
}
1 change: 1 addition & 0 deletions engine/runner/src/main/java/module-info.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
requires org.enso.logging.config;
requires org.enso.logging.utils;
requires org.enso.runtime.parser;
requires org.enso.runtime.version.manager;
requires org.enso.runner.common;
requires org.enso.pkg;
requires org.enso.polyglot.api;
Expand Down
110 changes: 110 additions & 0 deletions engine/runner/src/main/java/org/enso/runner/JavaFinder.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
package org.enso.runner;

import java.io.IOException;
import java.nio.file.Path;
import java.util.Comparator;
import java.util.concurrent.TimeUnit;
import org.enso.distribution.DistributionManager;
import org.enso.distribution.Environment;
import org.enso.runtimeversionmanager.components.GraalRuntime;
import org.enso.runtimeversionmanager.components.GraalVMVersion;
import org.enso.runtimeversionmanager.components.GraalVersionManager;
import org.enso.version.BuildVersion;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/** Utility class that tries to find installed JDK on the system. */
final class JavaFinder {
private static final Logger logger = LoggerFactory.getLogger(JavaFinder.class);

private JavaFinder() {}

/**
* Tries to find {@code java} executable on the system. If a system-wide JDK is not found, tries
* to find it in the {@link DistributionManager distribution} runtimes.
*
* @return null if cannot be found. Otherwise, returns the absolute path to the executable, or
* simply {@code java} if it is on the {@code PATH}.
*/
static String findJavaExecutable() {
var javaInRuntime = findJavaExecutableInDistributionRuntimes();
if (javaInRuntime != null) {
return javaInRuntime.toAbsolutePath().toString();
}
logger.warn("No appropriate JDK found in the distribution runtimes. Trying system-wide JDK.");
var javaHome = System.getenv("JAVA_HOME");
if (javaHome != null) {
var binDir = Path.of(javaHome).resolve("bin");
Path javaExe;
if (isOnWindows()) {
javaExe = binDir.resolve("java.exe");
} else {
javaExe = binDir.resolve("java");
}
if (javaExe.toFile().exists()) {
logger.warn("Found JDK in JAVA_HOME: {}", javaHome);
return javaExe.toAbsolutePath().toString();
}
}
logger.warn("No JDK found in JAVA_HOME. Trying java on PATH.");
if (isJavaOnPath()) {
var javaExe = isOnWindows() ? "java.exe" : "java";
logger.warn("Falling back to java on PATH: {}", javaExe);
return javaExe;
}
logger.warn("No JDK found on PATH. Cannot start the runtime.");
return null;
}

private static boolean isOnWindows() {
return System.getProperty("os.name").equals("windows");
}

/**
* Tries to find {@code java} executable in the distribution runtime with the same version that
* was used for building, or a newer one.
*
* @return null if not found.
*/
private static Path findJavaExecutableInDistributionRuntimes() {
var env = new Environment() {};
var distributionManager = new DistributionManager(env);
var graalVersionManager = new GraalVersionManager(distributionManager, env);
var versionUsedForBuild =
new GraalVMVersion(BuildVersion.graalVersion(), BuildVersion.javaVersion());
var runtimeWithExactVersionMatch = graalVersionManager.findGraalRuntime(versionUsedForBuild);
if (runtimeWithExactVersionMatch != null) {
return runtimeWithExactVersionMatch.javaExecutable();
}
// Try to find newer runtime (JDK).
var newerRuntime =
graalVersionManager.getAllRuntimes().stream()
.sorted(Comparator.comparing(GraalRuntime::version))
.filter(runtime -> runtime.version().compareTo(versionUsedForBuild) > 0)
.findFirst();
if (newerRuntime.isPresent()) {
logger.warn(
"Found newer JDK [{}] than the one used for build [{}]",
newerRuntime.get().version(),
versionUsedForBuild);
return newerRuntime.get().javaExecutable();
}
return null;
}

private static boolean isJavaOnPath() {
try {
ProcessBuilder processBuilder;
if (isOnWindows()) {
processBuilder = new ProcessBuilder("java.exe", "-h");
} else {
processBuilder = new ProcessBuilder("java", "-h");
}
Process process = processBuilder.start();
boolean exitSucc = process.waitFor(5L, TimeUnit.SECONDS);
return exitSucc;
} catch (IOException | InterruptedException e) {
return false;
}
}
}
19 changes: 5 additions & 14 deletions engine/runner/src/main/java/org/enso/runner/Main.java
Original file line number Diff line number Diff line change
Expand Up @@ -1339,22 +1339,13 @@ private void launch(String[] args) throws IOException, InterruptedException, URI
println(JVM_OPTION + " option has no effect - already running in JVM " + current);
} else {
var commandAndArgs = new ArrayList<String>();
JVM_FOUND:
if (jvm == null) {
var env = new Environment() {};
var dm = new DistributionManager(env);
var paths = dm.paths();
var files = paths.runtimes().toFile().listFiles();
if (files != null) {
for (var d : files) {
var java = new File(new File(d, "bin"), "java").getAbsoluteFile();
if (java.exists()) {
commandAndArgs.add(java.getPath());
break JVM_FOUND;
}
}
var javaExe = JavaFinder.findJavaExecutable();
if (javaExe == null) {
println("Cannot find java executable");
throw exitFail();
}
commandAndArgs.add("java");
commandAndArgs.add(javaExe);
} else {
commandAndArgs.add(new File(new File(new File(jvm), "bin"), "java").getAbsolutePath());
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,12 +52,6 @@ object DefaultDistributionConfiguration
lazy val temporaryDirectoryManager =
TemporaryDirectoryManager(distributionManager, resourceManager)

lazy val componentConfiguration: RuntimeComponentConfiguration =
new GraalVMComponentConfiguration

lazy val runtimeComponentUpdaterFactory: RuntimeComponentUpdaterFactory =
RuntimeComponentUpdaterFactory.Default

/** @inheritdoc */
def engineReleaseProvider: ReleaseProvider[EngineRelease] =
EngineRepository.defaultEngineReleaseProvider
Expand All @@ -67,15 +61,15 @@ object DefaultDistributionConfiguration
userInterface: RuntimeVersionManagementUserInterface
): RuntimeVersionManager =
new RuntimeVersionManager(
environment = this.environment,
userInterface = userInterface,
distributionManager = distributionManager,
environment = this.environment,
userInterface = userInterface,
distributionManager = distributionManager,
graalVersionManager =
new GraalVersionManager(distributionManager, environment),
temporaryDirectoryManager = temporaryDirectoryManager,
resourceManager = resourceManager,
engineReleaseProvider = engineReleaseProvider,
runtimeReleaseProvider = GraalCEReleaseProvider.default,
componentConfig = componentConfiguration,
componentUpdaterFactory = runtimeComponentUpdaterFactory,
installerKind = InstallerKind.ProjectManager
)

Expand Down
Loading
Loading