Skip to content

Commit

Permalink
Fix the failing exit handler SDK execution tests
Browse files Browse the repository at this point in the history
The SDK execution tests were not run on the PR CI that fixed the
pipeline run state to be failed even when an exit handler is run after a
failed component.

This also makes the SDK execution tests run whenever the backend
compiler code is modified to help prevent a similar issue.

Signed-off-by: mprahl <mprahl@users.noreply.github.com>
  • Loading branch information
mprahl committed Jan 15, 2025
1 parent 028d81b commit 56d1142
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 1 deletion.
1 change: 1 addition & 0 deletions .github/workflows/sdk-execution.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
- '.github/resources/**'
- 'sdk/python/**'
- 'api/v2alpha1/**'
- 'backend/src/v2/compiler/**'

jobs:
sdk-execution-tests:
Expand Down
2 changes: 2 additions & 0 deletions sdk/python/test_data/test_data_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ pipelines:
- module: pipeline_with_exit_handler
name: my_pipeline
execute: true
expected_state: FAILED
- module: pipeline_with_env
name: my_pipeline
execute: true
Expand Down Expand Up @@ -114,6 +115,7 @@ pipelines:
- module: pipeline_with_multiple_exit_handlers
name: my_pipeline
execute: true
expected_state: FAILED
- module: pipeline_with_parallelfor_parallelism
name: my_pipeline
execute: false
Expand Down
4 changes: 3 additions & 1 deletion test/sdk-execution-tests/sdk_execution_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ class TestCase:
yaml_path: str
function_name: str
arguments: Dict[str, Any]
expected_state: str


def create_test_case_parameters() -> List[TestCase]:
Expand All @@ -60,6 +61,7 @@ def create_test_case_parameters() -> List[TestCase]:
f'{test_case["module"]}.yaml'),
function_name=test_case['name'],
arguments=test_case.get('arguments'),
expected_state=test_case.get('expected_state', 'SUCCEEDED'),
) for test_case in test_group['test_cases'] if test_case['execute'])

return parameters
Expand Down Expand Up @@ -122,7 +124,7 @@ async def test(test_case: TestCase) -> None:
f'Error triggering pipeline {test_case.name}.') from e

api_run = await event_loop.run_in_executor(None, wait, run_result)
assert api_run.state == 'SUCCEEDED', f'Pipeline {test_case.name} ended with incorrect status: {api_run.state}. More info: {run_url}'
assert api_run.state == test_case.expected_state, f'Pipeline {test_case.name} ended with incorrect status: {api_run.state}. More info: {run_url}'


if __name__ == '__main__':
Expand Down

0 comments on commit 56d1142

Please sign in to comment.