Skip to content

Commit

Permalink
Fix Dashboard Sanity and Tier1 Tests on RHOAI 2.12 (red-hat-data-serv…
Browse files Browse the repository at this point in the history 
…ices#1709)

* Fix Dashboard tests on RHOAI 2.12

Fix following Dashboard Keywords:
- Login To RHODS Dashboard
- Open ODS Dashboard With Non Admin User
- Verify Username Displayed On RHODS Dashboard
- Open Application Switcher Menu
- Add OpenShift Groups To Data Science Administrators
- Add OpenShift Groups To Data Science User Groups
- Set Expected Items Based On RHODS Type
- Open QuickStart Element in Resource Section By Name
- Page Should Not Contain QuickStart Sidebar
- Quickstart Steps URLs Are Valid
- Verify Links Are Not Broken For Each QuickStart

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix Common KW `Check HTTP Status Code`

The KW had false negative on verifying URLs.
For example, URL 'https://tutorial.dask.org/' returned '403'
when it should have not.
Retrying requests with empty Headers has fixed it.

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix more Dashboard KW and Tests for RHOAI 2.12

Improve timeouts for Keywords:
- Apply Access Groups Settings
- Navigate To Page
- Wait For RHODS Dashboard To Load
- Save Changes In User Management Setting

Fix Xpath for Keywords:
- Login To Openshift
- Logout From RHODS Dashboard
- Add OpenShift Groups To Data Science Administrators
- Add OpenShift Groups To Data Science User Groups

Fix Tests:
- Verify Unauthorized User Is Not Able To Spawn Jupyter Notebook

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix Test `Verify Custom Application Resource Creation`

Including fixes to:
- AppsInfoDictionary_latest.json data
- KW `Check Sidebar Header Text`
- KW `Close Get Started Sidebar`

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix Test `Verify User Can Edit A Workbench`

By fixing Xpath in KW `Click Action From Actions Menu`

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix Test that verifies 'Right Number Of Items The User Has Selected'

- Fix Xpaths
- Add a helper Keyword `Filter Projects By ${Type}`
  to search for projects where Name or User includes a specific text.
- Fix `Create Multiple Data Science Projects`
- Fix `Delete Multiple Data Science Projects`

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix Tests in 410__ods_dashboard_projects_notebook_images.robot

Fix tests:
- Verify User Can Create A Workbench Using Intel AiKit Image
- Verify User Can Create A Workbench Using Code Server Image

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix Tests in 410__ods_dashboard_projects_permissions_mgmt.robot

Fixes for the "Editor" role, that has changed to "Contributor" role.

Also fixes in Permissions.resource Keywords:
- Assign ${permission_type} Permissions To User ${username}
- Assign ${permission_type} Permissions To Group ${group_name}
- Select Permission Type

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix ODHDashboardSettings Test and Keywords

Fix Tests:
- Verify RHODS Accept Multiple Admin Groups And CRD Gets Updates
- Verify If Unauthorized User Can Not Change The Permission

Fix Keywords:
- Launch Dashboard And Check User Management Option Is Available
- Remove OpenShift Groups From Data Science User Groups
- Remove OpenShift Groups From Data Science Administrator Groups
- Save Changes In User Management Setting
- AdminGroups In OdhDashboardConfig CRD Should Be
- AllowedGroups In OdhDashboardConfig CRD Should Be

New Keywords:
- Settings Page Should Be Unavailable
- Lists Should Contain Same Items

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix Dashboard Tests for Jupiter and Accelerator

Fix Dashboard Tests:
- Verify User Can Access Only Its Owned Projects
- Verify User Can Log Out And Return To Project From Jupyter Notebook
- Create / Modify An Accelerator Profile From Accelerator

Signed-off-by: manosnoam <nmanos@redhat.com>

* Fix Linter syntax warnings

Signed-off-by: manosnoam <nmanos@redhat.com>

---------

Signed-off-by: manosnoam <nmanos@redhat.com>
  • Loading branch information
@manosnoam
manosnoam authored Aug 12, 2024
1 parent 7a313f6 commit e35aafe
Show file tree
Hide file tree
Showing 22 changed files with 270 additions and 184 deletions.
25 changes: 20 additions & 5 deletions ods_ci/tests/Resources/Common.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ Begin Web Test

End Web Test
[Arguments] ${username}=${TEST_USER.USERNAME}
${server}= Run Keyword and Return Status Page Should Contain Element //div[@id='jp-top-panel']//div[contains(@class, 'p-MenuBar-itemLabel')][text() = 'File']
${server}= Run Keyword And Return Status Page Should Contain Element
... //div[@id='jp-top-panel']//div[contains(@class, 'p-MenuBar-itemLabel')][text() = 'File']
IF ${server}==True
Clean Up Server username=${username}
Stop JupyterLab Notebook Server
Expand Down Expand Up @@ -128,6 +129,13 @@ Get All Strings That Contain
END
RETURN ${matched_list}

Lists Should Contain Same Items
[Documentation] Compare two lists, but ignore the order of the items
[Arguments] ${list_one} ${list_two}
Sort List ${list_one}
Sort List ${list_two}
Lists Should Be Equal ${list_one} ${list_two}

Page Should Contain A String In List
[Documentation] Verifies that page contains at least one of the strings in text_list
[Arguments] ${text_list}
Expand Down Expand Up @@ -213,10 +221,17 @@ Wait Until HTTP Status Code Is

Check HTTP Status Code
[Documentation] Verifies Status Code of URL Matches Expected Status Code
[Arguments] ${link_to_check} ${expected}=200 ${timeout}=20 ${verify_ssl}=${True}
${headers}= Create Dictionary User-Agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36
${response}= RequestsLibrary.GET ${link_to_check} expected_status=any headers=${headers} timeout=${timeout} verify=${verify_ssl}
Run Keyword And Continue On Failure Status Should Be ${expected}
[Arguments] ${link_to_check} ${expected}=200 ${timeout}=20 ${verify_ssl}=${True} ${allow_redirects}=${True}
${headers}= Create Dictionary User-Agent="Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0"
${response}= RequestsLibrary.GET ${link_to_check} expected_status=any headers=${headers}
... timeout=${timeout} verify=${verify_ssl} allow_redirects=${allow_redirects}
${status_verified}= Run Keyword And Return Status Status Should Be ${expected} ${response}
IF not ${status_verified}
Log URL '${link_to_check}' returned '${response.status_code}' - Retrying with empty Headers console=True
${response}= RequestsLibrary.GET ${link_to_check} expected_status=any
... timeout=${timeout} verify=${verify_ssl} allow_redirects=${allow_redirects}
Run Keyword And Continue On Failure Status Should Be ${expected} ${response}
END
RETURN ${response.status_code}

URLs HTTP Status Code Should Be Equal To
Expand Down
6 changes: 3 additions & 3 deletions ods_ci/tests/Resources/Files/AppsInfoDictionary_latest.json
View file

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions ods_ci/tests/Resources/ODS.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,9 @@ Set Standard RHODS Groups Variables
Apply Access Groups Settings
[Documentation] Changes the rhods-groups config map to set the new access configuration
... and rolls out JH to make the changes effecting in Jupyter
[Arguments] ${admins_group} ${users_group}
[Arguments] ${admins_group} ${users_group} ${timeout}=10s
Set Access Groups Settings admins_group=${admins_group} users_group=${users_group}
Sleep 120 reason=Wait for Dashboard to get the updated configuration...
Sleep ${timeout} reason=Wait for Dashboard to get the updated configuration...

Set Access Groups Settings
[Documentation] Changes the rhods-groups config map to set the new access configuration
Expand Down
3 changes: 2 additions & 1 deletion ods_ci/tests/Resources/Page/Components/Menu.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ Navigate To Page
[Arguments]
... ${menu}
... ${submenu}=${NONE}
Wait Until Page Contains ${menu} timeout=150
... ${timeout}=10s
Wait Until Page Contains ${menu} timeout=${timeout}
IF "${submenu}" == "${NONE}" Run Keyword And Return
... Click Link ${menu}
${is_menu_expanded}= Menu.Is Menu Expanded ${menu}
Expand Down
35 changes: 22 additions & 13 deletions ods_ci/tests/Resources/Page/LoginPage.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,26 @@ Library JupyterLibrary
Is ${service_account_name} Service Account Authorization Required
${title} = Get Title
${result} = Run Keyword And Return Status Should Start With ${title} Authorize service account ${service_account_name}
[Return] ${result}
RETURN ${result}

Does Login Require Authentication Type
${authentication_required} = Run Keyword and Return Status Page Should Contain Log in with
[Return] ${authentication_required}
${authentication_required} = Run Keyword And Return Status Page Should Contain Log in with
RETURN ${authentication_required}

Is OpenShift OAuth Login Prompt Visible
${login_prompt_visible} = Run Keyword and Return Status Page Should Contain Log in with
${oauth_login} = Run Keyword and Return Status Page Should Contain oauth
${login_prompt_visible} = Run Keyword And Return Status Page Should Contain Log in with
${oauth_login} = Run Keyword And Return Status Page Should Contain oauth
${result} = Evaluate ${login_prompt_visible} and ${oauth_login}
[Return] ${result}
RETURN ${result}

Is OpenShift Login Visible
${login_prompt_visible} = Run Keyword and Return Status Page Should Contain Log in with
IF ${login_prompt_visible} RETURN True
${login_prompt_visible} = Run Keyword and Return Status Page Should Contain Log in to your account
[Return] ${login_prompt_visible}
[Arguments] ${timeout}=15s
${login_prompt_visible} = Run Keyword And Return Status
... Wait Until Page Contains Log in with timeout=${timeout}
IF ${login_prompt_visible} RETURN ${TRUE}
${login_prompt_visible} = Run Keyword And Return Status
... Wait Until Page Contains Log in to your account timeout=${timeout}
RETURN ${login_prompt_visible}

Select Login Authentication Type
[Arguments] ${auth_type}
Expand Down Expand Up @@ -54,15 +57,21 @@ Login To Openshift
Wait Until Page Contains Log in to your account
Input Text id=inputUsername ${ocp_user_name}
Input Text id=inputPassword ${ocp_user_pw}
Click Element xpath=/html/body/div/div/main/div/form/div[4]/button
Click Button //*[@type="submit"]
Maybe Skip Tour

Log In Should Be Requested
[Documentation] Passes if the login page appears and fails otherwise
${present} = Is OpenShift Login Visible
IF ${present} == ${FALSE} Fail msg=Log in page did not appear as expected
IF not ${present}
Capture Page Screenshot
Fail msg=Login page did not appear as expected
END

Log In Should Not Be Requested
[Documentation] Fails if the login page appears and passes otherwise
${present} = Is OpenShift Login Visible
IF ${present} == ${TRUE} Fail msg=Log in page did not appear as expected
IF ${present}
Capture Page Screenshot
Fail msg=Login page appeared but it was not expected
END
1 change: 1 addition & 0 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDashboard.resource
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@ Open ODS Dashboard With Non Admin User
[Documentation] Opens a browser and logs into ODS Dashboard with a user belonging to the rhods-users group
Launch Dashboard ${TEST_USER_3.USERNAME} ${TEST_USER_3.PASSWORD} ${TEST_USER_3.AUTH_TYPE}
... ${ODH_DASHBOARD_URL} browser=${BROWSER.NAME} browser_options=${BROWSER.OPTIONS}
... expected_page=${NONE}
61 changes: 27 additions & 34 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDashboard.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ ${NOTIFICATION_DRAWER_CLOSED}= //div[@class="pf-v5-c-drawer__panel" and @hidden
${GROUPS_CONFIG_CM}= groups-config
${RHODS_GROUPS_CONFIG_CM}= rhods-groups-config
${RHODS_LOGO_XPATH}= //img[@alt="${ODH_DASHBOARD_PROJECT_NAME} Logo"]
${USER_MENU_TOGGLE}= //button[@id="user-menu-toggle"]
${LOGOUT_BTN}= //button[.="Log out"]
@{ISV_TO_REMOVE_SELF_MANAGED}= Create List starburst nvidia rhoam


Expand All @@ -74,7 +76,7 @@ Authorize rhods-dashboard service account
Login To RHODS Dashboard
[Arguments] ${ocp_user_name} ${ocp_user_pw} ${ocp_user_auth_type}
# Wait until we are in the OpenShift auth page or already in Dashboard
${expected_text_list}= Create List Log in with Data Science Projects
${expected_text_list}= Create List Log in with
Wait Until Page Contains A String In List ${expected_text_list}
${oauth_prompt_visible}= Is OpenShift OAuth Login Prompt Visible
IF ${oauth_prompt_visible} Click Button Log in with OpenShift
Expand All @@ -88,25 +90,20 @@ Logout From RHODS Dashboard
[Documentation] Logs out from the current user in the RHODS dashboard
... This will reload the page and show the `Log in with OpenShift` page
... so you want to use `Login to RHODS Dashboard` after this
# Another option for the logout button
#${user} = Get Text xpath:/html/body/div/div/header/div[2]/div/div[3]/div/button/span[1]
#Click Element xpath://span[.="${user}"]/..
${version_check}= Is RHODS Version Greater Or Equal Than 1.21.0
IF ${version_check}==True
Click Button xpath://button[@id="user-menu-toggle"]
ELSE
Click Button xpath:(//button[@id="toggle-id"])[2]
END
Wait Until Page Contains Element xpath://a[.="Log out"]
Click Element xpath://a[.="Log out"]
Click Button ${USER_MENU_TOGGLE}
Wait Until Page Contains Element xpath:${LOGOUT_BTN}
Click Element xpath:${LOGOUT_BTN}
Wait Until Page Contains Log in with OpenShift

Wait For RHODS Dashboard To Load
[Arguments] ${dashboard_title}="${ODH_DASHBOARD_PROJECT_NAME}" ${wait_for_cards}=${TRUE}
... ${expected_page}=Enabled
Wait For Condition return document.title == ${dashboard_title} timeout=60s
Wait Until Page Contains Element xpath:${RHODS_LOGO_XPATH} timeout=20s
IF "${expected_page}" != "${NONE}" Wait For Dashboard Page Title ${expected_page} timeout=75s
... ${expected_page}=Enabled ${timeout}=60
${half_timeout}= Evaluate int(${timeout}) / 2
Wait For Condition return document.title == ${dashboard_title} timeout=${half_timeout}
Wait Until Page Contains Element xpath:${RHODS_LOGO_XPATH} timeout=${half_timeout}
IF "${expected_page}" != "${NONE}"
Wait For Dashboard Page Title ${expected_page} timeout=${timeout}
END
IF ${wait_for_cards} == ${TRUE}
Wait Until Keyword Succeeds 3 times 5 seconds Wait Until Cards Are Loaded
END
Expand Down Expand Up @@ -181,7 +178,7 @@ Verify Service Is Available In The Explore Page
END

Verify Service Is Not Available In The Explore Page
[Documentation] Verify the service appears in Applications > Explore
[Documentation] Verify the service does not appear in Applications > Explore
[Arguments] ${app_name} ${split_last}=${FALSE}
Menu.Navigate To Page Applications Explore
Wait For RHODS Dashboard To Load expected_page=Explore
Expand Down Expand Up @@ -263,7 +260,7 @@ Load Expected Data Of RHODS Explore Section

Wait Until Cards Are Loaded
[Documentation] Waits until the Application cards are displayed in the page
${status}= Run Keyword and Return Status Wait Until Page Contains Element
${status}= Run Keyword And Return Status Wait Until Page Contains Element
... xpath:${CARDS_XP} timeout=10s
IF not ${status} Reload Page
Should Be True ${status} msg=This might be caused by bug RHOAIENG-404
Expand Down Expand Up @@ -353,7 +350,7 @@ Open Get Started Sidebar And Return Status

Close Get Started Sidebar
Click Button xpath://button[@aria-label='Close drawer panel']
Wait Until Page Does Not Contain Element xpath://div[contains(@class,'odh-markdown-view')]/h1
Wait Until Page Does Not Contain Element xpath:${EXPLORE_PANEL_XP}

Check Get Started Sidebar Status
[Arguments] ${sidebar_status} ${badges_titles}
Expand Down Expand Up @@ -398,8 +395,10 @@ Check Sidebar Links

Check Sidebar Header Text
[Arguments] ${app_id} ${expected_data}
${h1}= Get Text xpath://div[contains(@class,'odh-markdown-view')]/h1
Run Keyword And Continue On Failure Should Be Equal ${h1} ${expected_data}[${app_id}][sidebar_h1]
Page Should Contain Element xpath:${SIDEBAR_TEXT_CONTAINER_XP}/h1
... message=Missing Sidebar Title for App Card ${app_id}
${header}= Get Text xpath:${SIDEBAR_TEXT_CONTAINER_XP}/h1
Run Keyword And Continue On Failure Should Be Equal ${header} ${expected_data}[${app_id}][sidebar_h1]
${getstarted_title}= Get Text xpath://div[contains(@class,'pf-v5-c-drawer__head')]
${titles}= Split String ${getstarted_title} separator=\n max_split=1
Run Keyword And Continue On Failure Should Be Equal ${titles[0]} ${expected_data}[${app_id}][title]
Expand Down Expand Up @@ -541,15 +540,7 @@ Search Items In Resources Section
Verify Username Displayed On RHODS Dashboard
[Documentation] Verifies that given username matches with username present on RHODS Dashboard
[Arguments] ${user_name}
${version_check}= Is RHODS Version Greater Or Equal Than 1.21.0
IF ${version_check}==True
${versioned_user_xp}= Set Variable
... xpath=//button[@id="user-menu-toggle"]/span[contains(@class,'toggle-text')]
ELSE
${versioned_user_xp}= Set Variable xpath=//div[@class='pf-v5-c-page__header-tools-item'][3]//span[1]
END

Element Text Should Be ${versioned_user_xp} ${user_name}
Element Text Should Be ${USER_MENU_TOGGLE} ${user_name}

RHODS Notification Drawer Should Contain
[Documentation] Verifies RHODS Notifications contains given Message
Expand Down Expand Up @@ -807,7 +798,7 @@ Get Links From Switcher

Open Application Switcher Menu
[Documentation] Clicks on the App Switcher in the top navigation bar of RHODS Dashboard
Click Button //button[@class="pf-v5-c-app-launcher__toggle"]
Click Button //div[@data-testid="application-launcher"]//button

Maybe Wait For Dashboard Loading Spinner Page
[Documentation] Detecs the loading symbol (spinner) and wait for it to disappear.
Expand Down Expand Up @@ -856,10 +847,12 @@ Handle Deletion Confirmation Modal
Click Action From Actions Menu
[Documentation] Clicks an action from Actions menu (3-dots menu on the right)
[Arguments] ${item_title} ${action} ${item_type}=${NONE}
Click Element xpath=//tr[td[@data-label="Name"]//*[text()="${item_title}"]]/td[contains(@class,"-table__action")]//button[@aria-label="Kebab toggle"] # robocop: disable
${item_row}= Set Variable //tr[td[@data-label="Name"]//*[text()="${item_title}"]]
Click Element xpath=${item_row}//button[@aria-label="Kebab toggle"]
IF "${item_type}" != "${NONE}"
${action}= Catenate ${action} ${item_type}
END
Wait Until Page Contains Element xpath=//tr[td[@data-label="Name"]//*[text()="${item_title}"]]//td//li//*[text()="${action}"] # robocop: disable
Wait Until Page Contains Element
... xpath=${item_row}//button[@role="menuitem"]//*[.="${action}"]
Sleep 0.5 msg=Avoid element missclicking
Click Element xpath=//tr[td[@data-label="Name"]//*[text()="${item_title}"]]//td//li//*[text()="${action}"]
Click Element xpath=${item_row}//button[@role="menuitem"]//*[.="${action}"]
25 changes: 10 additions & 15 deletions ods_ci/tests/Resources/Page/ODH/ODHDashboard/ODHDashboardResources.resource
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Resource ../../../Common.robot
... https://user-dev-rhoam-quarkus-openapi-3scale-api.cluster.com/status/?user_key=.
... https://console-openshift-console.apps.test-cluster.example.com
... console-openshift-console.apps.test-cluster.example.com

${QUICKSTART_PANNEL}= //div[@data-test="quickstart drawer"]

*** Keywords ***
Open QuickStart Element in Resource Section By Name
Expand All @@ -22,7 +22,7 @@ Open QuickStart Element in Resource Section By Name
... //div[@id="${element}"]//button[contains(., 'Open') or contains(., 'Continue') or contains(., 'Restart')]
Wait Until Page Contains Element ${start_tutorial_btn_xp}
Click Button ${start_tutorial_btn_xp}
Wait Until Page Contains Element //div[@data-test="quickstart drawer"] 5
Wait Until Page Contains Element ${QUICKSTART_PANNEL} 5

Star QuickStart Tour
Click Button //button[@data-test="Start button"]
Expand Down Expand Up @@ -69,7 +69,7 @@ Close QuickStart From Top
IF ${btn_exists} Click Element //*[@data-test="${decision} button"]

Page Should Not Contain QuickStart Sidebar
Wait Until Page Does Not Contain Element //div[@class="pf-c-drawer__panel-main"] 5
Wait Until Page Does Not Contain Element ${QUICKSTART_PANNEL} 5

Close QuickStart From Button
Click Element //*[@data-testid="qs-drawer-close"]
Expand Down Expand Up @@ -224,19 +224,14 @@ Verify Quick Starts Work As Expected When At Least One Step Is Skipped
END
END

External URLs Should Not Be Broken
Quickstart Steps URLs Are Valid
[Documentation] Go through a QuickStart and checks the status of all the external links
${quick_start_steps}= Get WebElements //button[@class='pf-c-wizard__nav-link']
${element_list}= Get WebElements xpath=//div[@Class="pf-c-drawer__panel-main"]//a[@href]
URLs HTTP Status Code Should Be Equal To ${element_list}
${quick_start_steps}= Get WebElements //button[contains(@class, "nav-link")]
FOR ${quick_start_step} IN @{quick_start_steps}
Open QuickStart Step ${quick_start_step}
${element_list}= Get WebElements xpath=//div[@Class="pf-c-drawer__panel-main"]//a[@href]
Run Keyword And Continue On Failure
... URLs HTTP Status Code Should Be Equal To ${element_list}
${doc_text}= Get Text //*[@class="pf-c-drawer__body pf-m-no-padding pfext-quick-start-panel-content__body"] # robocop: disable
${doc_link}= Extract URLs From Text text=${doc_text}
Validate Links Extracted From Text ${doc_link}
Open QuickStart Step ${quick_start_step}
${doc_text}= Get Text //*[@class="pfext-quick-start-task__content"]
${doc_link}= Extract URLs From Text text=${doc_text}
Validate Links Extracted From Text ${doc_link}
END

Verify Links Are Not Broken For Each QuickStart
Expand All @@ -257,7 +252,7 @@ Verify Links Are Not Broken For Each QuickStart
Log ${Title[${0}]} console=yes
Click Element ${quickStartElements[${counter}]}
Wait Until Element Is Visible //button[contains(@class, "-c-wizard__nav-link")]
External URLs Should Not Be Broken
Quickstart Steps URLs Are Valid
END

Get QuickStart Items
Expand Down
Loading

0 comments on commit e35aafe

Please sign in to comment.