Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport upgrade fixes #2136

Merged
merged 1 commit into from
Dec 16, 2024
Merged

Backport upgrade fixes #2136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 24 additions & 16 deletions ods_ci/tests/Tests/0200__rhoai_upgrade/0201__pre_upgrade.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,18 +31,9 @@ Test Tags PreUpgrade


*** Variables ***
${CUSTOM_CULLER_TIMEOUT} 60000
${S_SIZE} 25
${INFERENCE_INPUT}= @tests/Resources/Files/modelmesh-mnist-input.json
${INFERENCE_INPUT_OPENVINO}= @tests/Resources/Files/openvino-example-input.json
${EXPECTED_INFERENCE_OUTPUT}= {"model_name":"test-model__isvc-83d6fab7bd","model_version":"1","outputs":[{"name":"Plus214_Output_0","datatype":"FP32","shape":[1,10],"data":[-8.233053,-7.7497034,-3.4236815,12.3630295,-12.079103,17.266596,-10.570976,0.7130762,3.321715,1.3621228]}]}
${EXPECTED_INFERENCE_OUTPUT_OPENVINO}= {"model_name":"test-model__isvc-8655dc7979","model_version":"1","outputs":[{"name":"Func/StatefulPartitionedCall/output/_13:0","datatype":"FP32","shape":[1,1],"data":[0.99999994]}]}
${PRJ_TITLE}= model-serving-upgrade
${PRJ_DESCRIPTION}= project used for model serving tests
${MODEL_NAME}= test-model
${MODEL_CREATED}= ${FALSE}
${RUNTIME_NAME}= Model Serving Test
${DW_PROJECT_CREATED}= False
${CUSTOM_CULLER_TIMEOUT} 60000
${S_SIZE} 25
${DW_PROJECT_CREATED} False


*** Test Cases ***
Expand Down Expand Up @@ -97,10 +88,25 @@ Verify User Can Disable The Runtime
Disable Model Serving Runtime Using CLI namespace=redhat-ods-applications

Verify Model Can Be Deployed Via UI For Upgrade
[Tags] Upgrade
[Setup] Begin Web Test
${runtime_pod_name} = Replace String Using Regexp string=${RUNTIME_NAME} pattern=\\s replace_with=-
${runtime_pod_name} = Convert To Lower Case ${runtime_pod_name}
# robocop: off=too-long-test-case
# robocop: off=too-many-calls-in-test-case
[Documentation] Verify Model Can Be Deployed Via UI For Upgrade
[Tags] Upgrade
[Setup] Begin Web Test
${PRJ_TITLE}= Set Variable model-serving-upgrade
${PRJ_DESCRIPTION}= Set Variable project used for model serving tests
${MODEL_NAME}= Set Variable test-model
${MODEL_CREATED}= Set Variable ${FALSE}
${RUNTIME_NAME}= Set Variable Model Serving Test
${INFERENCE_INPUT_OPENVINO}= Set Variable
... @tests/Resources/Files/openvino-example-input.json
${EXPECTED_INFERENCE_OUTPUT_OPENVINO}= Set Variable
... {"model_name":"test-model__isvc-8655dc7979","model_version":"1","outputs":[{"name":"Func/StatefulPartitionedCall/output/_13:0","datatype":"FP32","shape":[1,1],"data":[0.99999994]}]} # robocop: disable:line-too-long
${runtime_pod_name}= Replace String Using Regexp
... string=${RUNTIME_NAME}
... pattern=\\s
... replace_with=-
${runtime_pod_name}= Convert To Lower Case ${runtime_pod_name}
Fetch CA Certificate If RHODS Is Self-Managed
Clean All Models Of Current User
Open Data Science Projects Home Page
Expand Down Expand Up @@ -128,11 +134,13 @@ Verify Model Can Be Deployed Via UI For Upgrade
... 10 sec
... Verify Openvino Deployment
... runtime_name=${runtime_pod_name}
... project_name=${PRJ_TITLE}
Run Keyword And Continue On Failure
... Wait Until Keyword Succeeds
... 5 min
... 10 sec
... Verify Serving Service
... project_name=${PRJ_TITLE}
Verify Model Status ${MODEL_NAME} success
Set Suite Variable ${MODEL_CREATED} ${TRUE} # robocop: disable:replace-set-variable-with-var
Run Keyword And Continue On Failure
Expand Down
27 changes: 14 additions & 13 deletions ods_ci/tests/Tests/0200__rhoai_upgrade/0203__post_upgrade.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,8 @@ Test Tags PostUpgrade


*** Variables ***
${S_SIZE} 25
${INFERENCE_INPUT}= @tests/Resources/Files/modelmesh-mnist-input.json
${INFERENCE_INPUT_OPENVINO}= @tests/Resources/Files/openvino-example-input.json
${EXPECTED_INFERENCE_OUTPUT}= {"model_name":"test-model__isvc-83d6fab7bd","model_version":"1","outputs":[{"name":"Plus214_Output_0","datatype":"FP32","shape":[1,10],"data":[-8.233053,-7.7497034,-3.4236815,12.3630295,-12.079103,17.266596,-10.570976,0.7130762,3.321715,1.3621228]}]}
${EXPECTED_INFERENCE_OUTPUT_OPENVINO}= {"model_name":"test-model__isvc-8655dc7979","model_version":"1","outputs":[{"name":"Func/StatefulPartitionedCall/output/_13:0","datatype":"FP32","shape":[1,1],"data":[0.99999994]}]}
${PRJ_TITLE}= model-serving-upgrade
${PRJ_DESCRIPTION}= project used for model serving tests
${MODEL_NAME}= test-model
${MODEL_CREATED}= ${FALSE}
${RUNTIME_NAME}= Model Serving Test
${DW_PROJECT_CREATED}= False
${S_SIZE} 25
${DW_PROJECT_CREATED} False


*** Test Cases ***
Expand Down Expand Up @@ -149,8 +140,18 @@ Test Inference Post RHODS Upgrade
# robocop: off=too-many-calls-in-test-case
# robocop: off=too-long-test-case
[Documentation] Test the inference result after having deployed a model that requires Token Authentication
[Tags] Upgrade
[Setup] Begin Web Test
[Tags] Upgrade
[Setup] Begin Web Test
${PRJ_TITLE} Set Variable model-serving-upgrade
${PRJ_DESCRIPTION} Set Variable project used for model serving tests # robocop: off=unused-variable # robocop: disable:line-too-long
${MODEL_NAME} Set Variable test-model
${MODEL_CREATED} Set Variable ${FALSE} # robocop: off=unused-variable
${RUNTIME_NAME} Set Variable Model Serving Test # robocop: off=unused-variable
${INFERENCE_INPUT} Set Variable @tests/Resources/Files/modelmesh-mnist-input.json # robocop: off=unused-variable # robocop: disable:line-too-long
${INFERENCE_INPUT_OPENVINO} Set Variable
... @tests/Resources/Files/openvino-example-input.json
${EXPECTED_INFERENCE_OUTPUT_OPENVINO} Set Variable
... {"model_name":"test-model__isvc-8655dc7979","model_version":"1","outputs":[{"name":"Func/StatefulPartitionedCall/output/_13:0","datatype":"FP32","shape":[1,1],"data":[0.99999994]}]} # robocop: disable:line-too-long
Fetch CA Certificate If RHODS Is Self-Managed
Open Model Serving Home Page
Verify Model Status ${MODEL_NAME} success
Expand Down
Loading