Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs around optimization levels for tests. #1769

Merged
merged 1 commit into from
Dec 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 16 additions & 19 deletions tests/testsrc/README.md → tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
```
./gradlew test
```
Runs the MozillaSuiteTest (and the Test262Suite if installed) 3 times (at optimization levels -1, 0 and 9)
Results can be found in `./buildGradle/reports/tests/test/index.html`
Runs the MozillaSuiteTest, the Test262Suite if installed, and many other tests.
Results can be found in the various "...build/reports/tests" subdirectories of each module.

## Running the official ECMAScript Test Suite (test262)
The Rhino test source contains logic to additionally run the official [ECMAScript Test Suite](https://github.com/tc39/test262).
Expand All @@ -19,19 +19,6 @@ After doing so, the `./gradlew test` command will also execute all tests that ar
As Rhino isn't 100% compliant with the latest ECMAScript standard, there is a mechanism to define which tests to run/skip,
through the [test262.properties](test262.properties) file, the format of which is discussed in the [test262.properties format](#test262.properties-format) section

## Optimization levels
By default all tests are run 3 times, at optimization levels -1, 0 and 9.

This behavior can be changed through different means:
1. Quick disable (will run tests with optimization level -1)
```
./gradlew test -Dquick
```
2. Setting an explicit optimization level through the command line:
```
./gradlew test -DoptLevel=9
```

## Running a specific TestSuite
```
./gradlew test --tests org.mozilla.javascript.tests.Test262SuiteTest
Expand Down Expand Up @@ -77,10 +64,17 @@ The test262.properties file uses the Java Properties format, with the folder/.js
```

A Java Properties file can also have entire lines as comments, by prefixing the line with either `!` or `#`.
While the test262.properties file does support this (because it is a Java Properties file), such line comments will be lost when (re)generating the test262.properties file!
While the test262.properties file does support this (because it is a Java Properties file), such line comments will be
lost when (re)generating the test262.properties file!

## Updating the test262.properties file
While the [test262.properties](test262.properties) file could be manually updated, the tooling also comes with a mechanism to (re)generate the file based on the current revision of the test262 submodule and the results of running Test262SuiteTest against this revision on all standard optLevels (-1, 0 & 9)
While the [test262.properties](test262.properties) file could be manually updated, the tooling also comes with a mechanism to
(re)generate the file based on the current revision of the test262 submodule and the results of running
the Test262SuiteTest against this revision on all standard optLevels (-1, 0 & 9)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still mentions the old levels.


(Note that the logic to build and organize regenerating this file seems dependent on Java version -- it may be
helpful to make sure that Java 11 is the first JVM in your path before trying this if the result seems
very unusual.)

```
RHINO_TEST_JAVA_VERSION=11 ./gradlew test --tests org.mozilla.javascript.tests.Test262SuiteTest --rerun-tasks -DupdateTest262properties [-Dtest262properties=testsrc/myOwn.properties]
Expand All @@ -95,7 +89,9 @@ These defaults can be overridden by specifying a value for the `generateTest262p
- none: rollup, stats and unsupported all false
- [rollup][stats][unsupported]: the ones included will be true

Note: the tests must actually run for the .properties file to be updated. If gradle determines that nothing has changed since the last time the `test` command was run, it won't run the tests. The `--rerun-tasks` argument forces gradle to run all tests
Note: the tests must actually run for the .properties file to be updated. If gradle determines that nothing has
changed since the last time the `test` command was run, it won't run the tests. The `--rerun-tasks` argument forces
gradle to run all tests

## Running specific tests from the official ECMAScript Test Suite (test262)
The default setup for running the test262 test suite is defined in [test262.properties](test262.properties).
Expand All @@ -104,4 +100,5 @@ Another .properties file to use can be specified using the `test262properties` c
```
./gradlew test --tests org.mozilla.javascript.tests.Test262SuiteTest -Dtest262properties=testsrc/myOwn.properties
```
This allows the creation of a custom .properties file containing for example just the tests for one specific feature being implemented, which allows for (quickly) running just the tests for that specific feature
This allows the creation of a custom .properties file containing for example just the tests for one specific feature
being implemented, which allows for (quickly) running just the tests for that specific feature
7 changes: 0 additions & 7 deletions tests/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,6 @@ test {
jvmArgs += ['--add-opens', 'java.base/java.util=ALL-UNNAMED']
jvmArgs += ['--add-opens', 'java.base/java.lang=ALL-UNNAMED']

// Set to reduce number of tests by only running at one opt level
if (System.getProperty('quick') != null) {
systemProperty 'TEST_OPTLEVEL', -1
} else if (System.getProperty('optLevel')) {
systemProperty 'TEST_OPTLEVEL', System.getProperty('optLevel')
}

systemProperty 'test262properties', System.getProperty('test262properties')
if (System.getProperty('updateTest262properties') != null) {
if (System.getenv("RHINO_TEST_JAVA_VERSION") != "11") {
Expand Down
Loading