Skip to content

Commit 7c5596f

Browse files
zjgarveyzjgarvey
committed
add some sample reports. Allow saving reports and jsons to a folder
1 parent 89e4ae8 commit 7c5596f

File tree

7 files changed

+148
-1
lines changed

7 files changed

+148
-1
lines changed

‎alt_e2eshark/reports/test_add.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"test_add": "PASS",
3+
"test_add_bcast": "PASS",
4+
"test_add_uint8": "PASS"
5+
}

‎alt_e2eshark/reports/test_add.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
## Summary
2+
3+
|Stage|Count|
4+
|--|--|
5+
| Total | 3 |
6+
| PASS | 3 |
7+
| Numerics | 0 |
8+
| results-summary | 0 |
9+
| postprocessing | 0 |
10+
| compiled_inference | 0 |
11+
| native_inference | 0 |
12+
| construct_inputs | 0 |
13+
| compilation | 0 |
14+
| preprocessing | 0 |
15+
| import_model | 0 |
16+
| setup | 0 |
17+
18+
## Test Run Detail
19+
Test was run with the following arguments:
20+
Namespace(device='local-task', backend='llvm-cpu', iree_compile_args=None, mode='onnx-iree', torchtolinalg=False, stages=None, skip_stages=None, load_inputs=False, groups='all', test_filter='test_add', tolerance=None, verbose=True, rundirectory='test-run', no_artifacts=False, report=True, report_file='reports/test_add.md')
21+
22+
| Test | Exit Status | Notes |
23+
|--|--|--|
24+
| test_add | PASS | |
25+
| test_add_bcast | PASS | |
26+
| test_add_uint8 | PASS | |

‎alt_e2eshark/reports/test_conv.json

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"test_conv_with_autopad_same": "compilation",
3+
"test_conv_with_strides_and_asymmetric_padding": "PASS",
4+
"test_conv_with_strides_no_padding": "PASS",
5+
"test_conv_with_strides_padding": "PASS",
6+
"test_convinteger_with_padding": "PASS",
7+
"test_convinteger_without_padding": "PASS",
8+
"test_convtranspose": "PASS",
9+
"test_convtranspose_1d": "PASS",
10+
"test_convtranspose_3d": "PASS",
11+
"test_convtranspose_autopad_same": "compilation",
12+
"test_convtranspose_dilations": "PASS",
13+
"test_convtranspose_kernel_shape": "compilation",
14+
"test_convtranspose_output_shape": "compilation",
15+
"test_convtranspose_pad": "PASS",
16+
"test_convtranspose_pads": "PASS"
17+
}

‎alt_e2eshark/reports/test_conv.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
## Summary
2+
3+
|Stage|Count|
4+
|--|--|
5+
| Total | 15 |
6+
| PASS | 11 |
7+
| Numerics | 0 |
8+
| results-summary | 0 |
9+
| postprocessing | 0 |
10+
| compiled_inference | 0 |
11+
| native_inference | 0 |
12+
| construct_inputs | 0 |
13+
| compilation | 4 |
14+
| preprocessing | 0 |
15+
| import_model | 0 |
16+
| setup | 0 |
17+
18+
## Test Run Detail
19+
Test was run with the following arguments:
20+
Namespace(device='local-task', backend='llvm-cpu', iree_compile_args=None, mode='onnx-iree', torchtolinalg=False, stages=None, skip_stages=None, load_inputs=False, groups='all', test_filter='test_conv', tolerance=None, verbose=True, rundirectory='test-run', no_artifacts=False, report=True, report_file='reports/test_conv.md')
21+
22+
| Test | Exit Status | Notes |
23+
|--|--|--|
24+
| test_conv_with_autopad_same | compilation | |
25+
| test_conv_with_strides_and_asymmetric_padding | PASS | |
26+
| test_conv_with_strides_no_padding | PASS | |
27+
| test_conv_with_strides_padding | PASS | |
28+
| test_convinteger_with_padding | PASS | |
29+
| test_convinteger_without_padding | PASS | |
30+
| test_convtranspose | PASS | |
31+
| test_convtranspose_1d | PASS | |
32+
| test_convtranspose_3d | PASS | |
33+
| test_convtranspose_autopad_same | compilation | |
34+
| test_convtranspose_dilations | PASS | |
35+
| test_convtranspose_kernel_shape | compilation | |
36+
| test_convtranspose_output_shape | compilation | |
37+
| test_convtranspose_pad | PASS | |
38+
| test_convtranspose_pads | PASS | |

‎alt_e2eshark/reports/test_conv_and_add.json

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"test_add": "PASS",
3+
"test_add_bcast": "PASS",
4+
"test_add_uint8": "PASS",
5+
"test_conv_with_autopad_same": "compilation",
6+
"test_conv_with_strides_and_asymmetric_padding": "PASS",
7+
"test_conv_with_strides_no_padding": "PASS",
8+
"test_conv_with_strides_padding": "PASS",
9+
"test_convinteger_with_padding": "PASS",
10+
"test_convinteger_without_padding": "PASS",
11+
"test_convtranspose": "PASS",
12+
"test_convtranspose_1d": "PASS",
13+
"test_convtranspose_3d": "PASS",
14+
"test_convtranspose_autopad_same": "compilation",
15+
"test_convtranspose_dilations": "PASS",
16+
"test_convtranspose_kernel_shape": "compilation",
17+
"test_convtranspose_output_shape": "compilation",
18+
"test_convtranspose_pad": "PASS",
19+
"test_convtranspose_pads": "PASS"
20+
}

‎alt_e2eshark/reports/test_conv_and_add.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
## Summary
2+
3+
|Stage|Count|
4+
|--|--|
5+
| Total | 18 |
6+
| PASS | 14 |
7+
| Numerics | 0 |
8+
| results-summary | 0 |
9+
| postprocessing | 0 |
10+
| compiled_inference | 0 |
11+
| native_inference | 0 |
12+
| construct_inputs | 0 |
13+
| compilation | 4 |
14+
| preprocessing | 0 |
15+
| import_model | 0 |
16+
| setup | 0 |
17+
18+
## Test Run Detail
19+
Test was run with the following arguments:
20+
Namespace(sources=['reports/test_add.json', 'reports/test_conv.json'], output='reports/test_conv_and_add.json', report=True, report_file='reports/test_conv_and_add.md')
21+
22+
| Test | Exit Status | Notes |
23+
|--|--|--|
24+
| test_add | PASS | |
25+
| test_add_bcast | PASS | |
26+
| test_add_uint8 | PASS | |
27+
| test_conv_with_autopad_same | compilation | |
28+
| test_conv_with_strides_and_asymmetric_padding | PASS | |
29+
| test_conv_with_strides_no_padding | PASS | |
30+
| test_conv_with_strides_padding | PASS | |
31+
| test_convinteger_with_padding | PASS | |
32+
| test_convinteger_without_padding | PASS | |
33+
| test_convtranspose | PASS | |
34+
| test_convtranspose_1d | PASS | |
35+
| test_convtranspose_3d | PASS | |
36+
| test_convtranspose_autopad_same | compilation | |
37+
| test_convtranspose_dilations | PASS | |
38+
| test_convtranspose_kernel_shape | compilation | |
39+
| test_convtranspose_output_shape | compilation | |
40+
| test_convtranspose_pad | PASS | |
41+
| test_convtranspose_pads | PASS | |

‎alt_e2eshark/run.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ def main(args):
108108

109109
if args.report:
110110
generate_report(args, stages, status_dict)
111-
json_save_to = Path(args.report_file).stem + ".json"
111+
json_save_to = str(Path(args.report_file).parent.joinpath(Path(args.report_file).stem + ".json"))
112112
save_dict(status_dict, json_save_to)
113113

114114

0 commit comments

Comments
 (0)