Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add web-tooling-benchmark test #1287

Merged
merged 2 commits into from
Dec 12, 2023
Merged

Add web-tooling-benchmark test #1287

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6aaedf5
Add web-tooling-benchmark test
clover2123 Dec 11, 2023
f9aacb6
Run web-tooling-benchmark in actions
clover2123 Dec 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 21 additions & 4 deletions .github/workflows/es-actions.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
tc: ['octane', 'v8 chakracore spidermonkey', 'jetstream-only-simple-parallel-1', 'jetstream-only-simple-parallel-2 jsc-stress', 'jetstream-only-simple-parallel-3 jetstream-only-cdjs']
tc: ['new-es octane', 'v8 chakracore spidermonkey', 'jetstream-only-simple-parallel-1', 'jetstream-only-simple-parallel-2 jsc-stress', 'jetstream-only-simple-parallel-3 jetstream-only-cdjs']
steps:
- uses: actions/checkout@v3
with:
Expand Down Expand Up @@ -304,11 +304,11 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
tc: ['new-es octane v8', 'chakracore spidermonkey']
tc: ['octane v8 web-tooling-benchmark', 'chakracore spidermonkey new-es']
build_opt: ['', '-DESCARGOT_THREADING=ON -DESCARGOT_TCO=ON', '-DESCARGOT_SMALL_CONFIG=ON -DESCARGOT_USE_CUSTOM_LOGGING=ON']
exclude:
# exclude octane, v8 due to low performance incurred by SMALL_CONFIG
- tc: 'new-es octane v8'
# exclude octane, v8, web-tooling-benchmark due to low performance incurred by SMALL_CONFIG
- tc: 'octane v8 web-tooling-benchmark'
build_opt: '-DESCARGOT_SMALL_CONFIG=ON -DESCARGOT_USE_CUSTOM_LOGGING=ON'
steps:
- uses: actions/checkout@v3
Expand Down Expand Up @@ -507,6 +507,15 @@ jobs:
export PKG_CONFIG_PATH=$GITHUB_WORKSPACE/icu64/usr/lib/x86_64-linux-gnu/pkgconfig
cmake -H. -Bout/codecache/x64 $BUILD_OPTIONS
ninja -Cout/codecache/x64
- name: Build x64 Release Mode
env:
BUILD_OPTIONS: -DESCARGOT_MODE=release -DESCARGOT_CODE_CACHE=ON -DESCARGOT_TEMPORAL=ON -DESCARGOT_TCO=ON -DESCARGOT_TEST=ON -DESCARGOT_OUTPUT=shell -GNinja
run: |
export CXXFLAGS="-I$GITHUB_WORKSPACE/icu64/usr/include"
export LDFLAGS="-L$GITHUB_WORKSPACE/icu64/usr/lib/x86_64-linux-gnu -Wl,-rpath=$GITHUB_WORKSPACE/icu64/usr/lib/x86_64-linux-gnu"
export PKG_CONFIG_PATH=$GITHUB_WORKSPACE/icu64/usr/lib/x86_64-linux-gnu/pkgconfig
cmake -H. -Bout/codecache/release/x64 $BUILD_OPTIONS
ninja -Cout/codecache/release/x64
- name: Run x86 test
run: |
$RUNNER --arch=x86 --engine="$GITHUB_WORKSPACE/out/codecache/x86/escargot" sunspider-js
Expand All @@ -525,6 +534,11 @@ jobs:
$RUNNER --arch=x86_64 --engine="$GITHUB_WORKSPACE/out/codecache/x64/escargot" octane-loading
$RUNNER --arch=x86_64 --engine="$GITHUB_WORKSPACE/out/codecache/x64/escargot" octane-loading
rm -rf $HOME/Escargot-cache/
- name: Run x64 release test
run: |
$RUNNER --arch=x86_64 --engine="$GITHUB_WORKSPACE/out/codecache/release/x64/escargot" web-tooling-benchmark
$RUNNER --arch=x86_64 --engine="$GITHUB_WORKSPACE/out/codecache/release/x64/escargot" web-tooling-benchmark
rm -rf $HOME/Escargot-cache/
- name: Handle error cases
run: |
$RUNNER --arch=x86_64 --engine="$GITHUB_WORKSPACE/out/codecache/x64/escargot" sunspider-js
Expand All @@ -534,6 +548,9 @@ jobs:
$RUNNER --arch=x86_64 --engine="$GITHUB_WORKSPACE/out/codecache/x64/escargot" sunspider-js
rm $HOME/Escargot-cache/cache_list
$RUNNER --arch=x86_64 --engine="$GITHUB_WORKSPACE/out/codecache/x64/escargot" sunspider-js
- if: ${{ failure() }}
uses: mxschmitt/action-tmate@v3
timeout-minutes: 15

build-test-wasmjs:
runs-on: ubuntu-latest
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,6 @@
path = third_party/walrus
url = https://github.com/Samsung/walrus.git
ignore = untracked
[submodule "test/web-tooling-benchmark"]
path = test/web-tooling-benchmark
url = https://github.com/v8/web-tooling-benchmark
1 change: 1 addition & 0 deletions test/web-tooling-benchmark
Open in desktop
Submodule web-tooling-benchmark added at 4a1282
31 changes: 29 additions & 2 deletions tools/run-tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -880,7 +880,7 @@ def run_escargot_debugger(engine, arch, extra_arg):
ESCARGOT_DEBUGGER_TESTER = join(PROJECT_SOURCE_DIR, 'tools', 'debugger', 'debugger_tester.sh')
print('Running Escargot-Debugger-Server-Source test:')
fails = 0
proc = Popen(['chmod', '+x', ESCARGOT_DEBUGGER_TESTER],stdout=PIPE)
proc = Popen(['chmod', '+x', ESCARGOT_DEBUGGER_TESTER], stdout=PIPE)
for files in os.listdir(ESCARGOT_DEBUGGER_TEST_DIR):
if files.endswith(".cmd"):
test_case, _ = os.path.splitext(files)
Expand All @@ -902,7 +902,7 @@ def run_escargot_debugger2(engine, arch, extra_arg):
ESCARGOT_DEBUGGER_TESTER = join(PROJECT_SOURCE_DIR, 'tools', 'debugger', 'debugger_tester.sh')
print('Running Escargot-Debugger-Client-Source test:')
fails = 0
proc = Popen(['chmod', '+x', ESCARGOT_DEBUGGER_TESTER],stdout=PIPE)
proc = Popen(['chmod', '+x', ESCARGOT_DEBUGGER_TESTER], stdout=PIPE)
for files in os.listdir(ESCARGOT_DEBUGGER_TEST_DIR):
if files.endswith(".cmd"):
test_case, _ = os.path.splitext(files)
Expand All @@ -917,6 +917,33 @@ def run_escargot_debugger2(engine, arch, extra_arg):
if fails > 0:
raise Exception('Escargot-Debugger-Client-Source tests failed')

@runner('web-tooling-benchmark', default=False)
def run_web_tooling_benchmark(engine, arch, extra_arg):
WEB_TOOLING_DIR = join(PROJECT_SOURCE_DIR, 'test', 'web-tooling-benchmark')
WEB_TOOLING_SRC_DIR = join(WEB_TOOLING_DIR, 'src')

suite_file = join(WEB_TOOLING_SRC_DIR, 'suite.js')
with open(suite_file, "r") as f:
lines = f.readlines()
with open(suite_file, "w") as f:
for line in lines:
# minimize sample count to 1
f.write(re.sub(r'minSamples: 20', 'minSamples: 1', line))

flag_file = join(WEB_TOOLING_SRC_DIR, 'cli-flags-helper.js')
with open(flag_file, "r") as f:
lines = f.readlines()
with open(flag_file, "w") as f:
for line in lines:
# exclude long running tests
if "babel" not in line and "babylon" not in line and "chai" not in line:
f.write(line)

proc = Popen(['npx', 'npm@6', 'install'], cwd=WEB_TOOLING_DIR, stdout=PIPE)
proc.wait()

run([engine, 'dist/cli.js'], cwd=WEB_TOOLING_DIR)

@runner('dump-all', default=False)
def run_dump_all(engine, arch, extra_arg):
for test in RUNNERS:
Expand Down
Loading