[Integration][Gitlab-V2] Handle HTTP error codes

[dev-habib-nuhu]

User description

Description

What - Implement graceful handling of HTTP error codes (401, 403, 404) in GitLab integration to prevent resync failures when accessing resources without proper permissions or when resources don't exist.

Why - Currently, the integration fails when encountering 401, 403, or 404 errors, particularly when trying to access pipelines or jobs for projects without CI/CD enabled. This causes the entire resync process to fail instead of gracefully skipping these resources.

How -

1. Added error handling in HTTPBaseClient to handle 401, 403, and 404 responses gracefully
2. Updated error handling to return empty responses instead of raising exceptions for these status codes
3. Added comprehensive tests to verify error handling behavior

Type of change

• Non-breaking change (fix of existing functionality that will not change current behavior)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

• Integration able to create all default resources from scratch
• Resync finishes successfully
• Resync able to create entities
• Resync able to update entities
• Resync able to detect and delete entities
• Scheduled resync able to abort existing resync and start a new one
• Tested with at least 2 integrations from scratch
• Tested with Kafka and Polling event listeners
• Tested deletion of entities that don't pass the selector

Integration testing checklist

• Integration able to create all default resources from scratch
• Resync able to create entities
• Resync able to update entities
• Resync able to detect and delete entities
• Resync finishes successfully
• If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
• If resource kind is updated, run the integration with the example data and check if the expected result is achieved
• If new resource kind is added or updated, validate that live-events for that resource are working as expected
• Docs PR link here

Preflight checklist

• Handled rate limiting
• Handled pagination
• Implemented the code in async
• Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.

---

PR Type

Bug fix, Tests

---

Description

• Gracefully handle 401, 403, and 404 errors in GitLab API client

  • Prevents resync failures due to missing permissions or resources
  • Returns empty responses instead of raising exceptions for these errors

• Enhance error handling in paginated resource fetching

  • Logs warnings and yields empty lists on fetch failures

• Add comprehensive tests for 401 and 403 error handling

---

Changes walkthrough 📝

	Relevant files
Bug fix	base_client.py Graceful handling of 401/403/404 errors in API requests    integrations/gitlab-v2/gitlab/clients/base_client.py Expanded error handling to treat 401, 403, and 404 as non-fatal Logs warnings and returns empty dict for these status codes +3/-2      rest_client.py Robust error handling in paginated and file resource fetching integrations/gitlab-v2/gitlab/clients/rest_client.py Wrapped paginated resource fetch in try/except to handle errors gracefully Logs warning and yields empty list on fetch failure Ensured file content decoding only occurs if response is present Minor formatting and logic improvements for clarity +20/-8
Tests	test_base_client.py Tests for 401 and 403 error handling in API client              integrations/gitlab-v2/tests/clients/test_base_client.py Added tests for 401 Unauthorized and 403 Forbidden error handling Validates that empty dict is returned for these errors +52/-0

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-merge-pro]

CI Feedback 🧐

(Feedback updated until commit eb19a25)

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: validate-files

Failed stage: Check Ocean version 🌊 [❌]

Failure summary: The action failed because the Ocean version in the integrations/gitlab-v2/pyproject.toml file is not updated to the latest version (0.24.2). The CI workflow checks if all changed integration directories have their port_ocean dependency version updated to match the latest PyPI version, and fails if any are outdated.

Relevant error logs: 1: ##[group]Runner Image Provisioner 2: Hosted Compute Agent ... 115: [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 116: [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 117: [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 118: ##[endgroup] 119: ##[group]Fetching the repository 120: [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* +3c11e56754a19a2710b9e138c180f6d52eb3369a:refs/remotes/pull/1735/merge 121: From https://github.com/port-labs/ocean 122: * [new branch] Github-tj-actions-bug-fix -> origin/Github-tj-actions-bug-fix 123: * [new branch] PORT-10639-add-memray-to-jira-integration -> origin/PORT-10639-add-memray-to-jira-integration 124: * [new branch] PORT-10680-Add-default-actions-for-Jira-integration -> origin/PORT-10680-Add-default-actions-for-Jira-integration 125: * [new branch] PORT-10680-Add-default-actions-for-PagerDuty-integration -> origin/PORT-10680-Add-default-actions-for-PagerDuty-integration 126: * [new branch] PORT-10776-Default-Action-for-Snyk-Ocean-Integration -> origin/PORT-10776-Default-Action-for-Snyk-Ocean-Integration 127: * [new branch] PORT-10933-bug-snyk-ingesting-vulnerabilities-from-wrong-org -> origin/PORT-10933-bug-snyk-ingesting-vulnerabilities-from-wrong-org 128: * [new branch] PORT-11183-fix-starlette-denial-of-service-do-s-via-multipart-form-data -> origin/PORT-11183-fix-starlette-denial-of-service-do-s-via-multipart-form-data 129: * [new branch] PORT-11220-automate-git-tag-when-pushing-a-new-commit-to-main -> origin/PORT-11220-automate-git-tag-when-pushing-a-new-commit-to-main 130: * [new branch] PORT-11682-ocean-handle-and-log-errors-of-client-timeouts-when-interacting-with-port-api-upserting-etc -> origin/PORT-11682-ocean-handle-and-log-errors-of-client-timeouts-when-interacting-with-port-api-upserting-etc 131: * [new branch] PORT-12105-Bug-Pagination-logic-error-in-the-SonarQube-integration -> origin/PORT-12105-Bug-Pagination-logic-error-in-the-SonarQube-integration 132: * [new branch] PORT-12472/gitlab-merge-request-event -> origin/PORT-12472/gitlab-merge-request-event ... 155: * [new branch] PORT-14515 -> origin/PORT-14515 156: * [new branch] PORT-14517 -> origin/PORT-14517 157: * [new branch] PORT-14520 -> origin/PORT-14520 158: * [new branch] PORT-14521 -> origin/PORT-14521 159: * [new branch] PORT-14554-Prepare-A-Script-To-Remove-Dangling-Entities -> origin/PORT-14554-Prepare-A-Script-To-Remove-Dangling-Entities 160: * [new branch] PORT-14640-shutting-down-gracefully-on-ocean-resync-exception -> origin/PORT-14640-shutting-down-gracefully-on-ocean-resync-exception 161: * [new branch] PORT-14823 -> origin/PORT-14823 162: * [new branch] PORT-14828 -> origin/PORT-14828 163: * [new branch] PORT-14837 -> origin/PORT-14837 164: * [new branch] PORT-4308-ocean-support-pydantic-v-2 -> origin/PORT-4308-ocean-support-pydantic-v-2 165: * [new branch] PORT-8504-add-configurable-limit-for-wiz-import-issues -> origin/PORT-8504-add-configurable-limit-for-wiz-import-issues 166: * [new branch] PORT-8596-add-the-ability-to-ingest-ami-images-and-acm-certificate-from-aws-new-integration-into-port -> origin/PORT-8596-add-the-ability-to-ingest-ami-images-and-acm-certificate-from-aws-new-integration-into-port 167: * [new branch] PORT-8788-Add-deployment-method-requirements-field -> origin/PORT-8788-Add-deployment-method-requirements-field 168: * [new branch] PORT-9072 -> origin/PORT-9072 169: * [new branch] PORT-9097-ocean-icons-are-not-synced -> origin/PORT-9097-ocean-icons-are-not-synced 170: * [new branch] PORT-9326-bug-sonarqube-integration-throws-errors -> origin/PORT-9326-bug-sonarqube-integration-throws-errors 171: * [new branch] PORT-9612-add-sonar-project-properties-file-to-ocean-integration-code-base -> origin/PORT-9612-add-sonar-project-properties-file-to-ocean-integration-code-base ... 236: * [new branch] feat-gitlabv2-dependencies-and-vulnerabilities -> origin/feat-gitlabv2-dependencies-and-vulnerabilities 237: * [new branch] feat-remove-polling-logs -> origin/feat-remove-polling-logs 238: * [new branch] feat/bitbucket -> origin/feat/bitbucket 239: * [new branch] feat/port-7767-support-exporting-jira-issues-by-sprint -> origin/feat/port-7767-support-exporting-jira-issues-by-sprint 240: * [new branch] feature-bump-single-integration -> origin/feature-bump-single-integration 241: * [new branch] fetch_and_jq -> origin/fetch_and_jq 242: * [new branch] fix-dev-komodor-integration -> origin/fix-dev-komodor-integration 243: * [new branch] fix-memory-leak-with-fastapi -> origin/fix-memory-leak-with-fastapi 244: * [new branch] fix-unexpected-lack-of-config -> origin/fix-unexpected-lack-of-config 245: * [new branch] fix/strip-app-host -> origin/fix/strip-app-host 246: * [new branch] format_url_types -> origin/format_url_types 247: * [new branch] gitlab-param-change -> origin/gitlab-param-change 248: * [new branch] gitlab-v2 -> origin/gitlab-v2 249: * [new branch] jira-oauth-support -> origin/jira-oauth-support 250: * [new branch] jortegac/main -> origin/jortegac/main 251: * [new branch] jq_next-assertion-error -> origin/jq_next-assertion-error 252: * [new branch] label-prs -> origin/label-prs 253: * [new branch] main -> origin/main 254: * [new branch] mapping_rule_schema_fix -> origin/mapping_rule_schema_fix 255: * [new branch] new_base_image -> origin/new_base_image 256: * [new branch] oauth-token-refresh-by-error-mode -> origin/oauth-token-refresh-by-error-mode 257: * [new branch] ocean_11_memo_leak -> origin/ocean_11_memo_leak ... 600: Downloading xmltodict-0.14.2-py2.py3-none-any.whl (10.0 kB) 601: Installing collected packages: xmltodict, typing-extensions, tomlkit, shellingham, regex, PyYAML, pygments, packaging, mdurl, click, argcomplete, yq, markdown-it-py, rich, typer, toml-cli 602: Successfully installed PyYAML-6.0.2 argcomplete-3.6.2 click-8.2.1 markdown-it-py-3.0.0 mdurl-0.1.2 packaging-25.0 pygments-2.19.1 regex-2024.11.6 rich-14.0.0 shellingham-1.5.4 toml-cli-0.7.0 tomlkit-0.13.2 typer-0.16.0 typing-extensions-4.14.0 xmltodict-0.14.2 yq-3.4.3 603: ##[group]Run git remote add ocean-origin https://github.com/port-labs/ocean.git 604: �[36;1mgit remote add ocean-origin https://github.com/port-labs/ocean.git�[0m 605: �[36;1mgit fetch ocean-origin�[0m 606: �[36;1mchanged_dirs=$(git diff --name-only ocean-origin/main HEAD | grep 'integrations/' | cut -d'/' -f 1,2 | sort -u)�[0m 607: �[36;1mpackage_version=$(curl -s https://pypi.org/pypi/port-ocean/json | jq -r '.info.version')�[0m 608: �[36;1mfor dir in $changed_dirs; do�[0m 609: �[36;1m pyproject_file=$(find $dir -name 'pyproject.toml' -not -path "**/.venv/*")�[0m 610: �[36;1m if [ -n "$pyproject_file" ]; then�[0m 611: �[36;1m installed_version=$(toml get tool.poetry.dependencies.port_ocean.version --toml-path $pyproject_file)�[0m 612: �[36;1m is_version_updated=$(python -c "from packaging import version;print(version.parse('$installed_version'.lstrip('^')) >= version.parse('$package_version'))")�[0m 613: �[36;1m�[0m 614: �[36;1m if [ "$is_version_updated" = "False" ]; then�[0m 615: �[36;1m echo "ERROR: Ocean version in $pyproject_file is not updated to latest version -> $package_version"�[0m 616: �[36;1m exit 1�[0m ... 625: PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.10/x64/lib/pkgconfig 626: Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.10/x64 627: Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.10/x64 628: Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.10/x64 629: LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.10/x64/lib 630: ##[endgroup] 631: From https://github.com/port-labs/ocean 632: * [new branch] Github-tj-actions-bug-fix -> ocean-origin/Github-tj-actions-bug-fix 633: * [new branch] PORT-10639-add-memray-to-jira-integration -> ocean-origin/PORT-10639-add-memray-to-jira-integration 634: * [new branch] PORT-10680-Add-default-actions-for-Jira-integration -> ocean-origin/PORT-10680-Add-default-actions-for-Jira-integration 635: * [new branch] PORT-10680-Add-default-actions-for-PagerDuty-integration -> ocean-origin/PORT-10680-Add-default-actions-for-PagerDuty-integration 636: * [new branch] PORT-10776-Default-Action-for-Snyk-Ocean-Integration -> ocean-origin/PORT-10776-Default-Action-for-Snyk-Ocean-Integration 637: * [new branch] PORT-10933-bug-snyk-ingesting-vulnerabilities-from-wrong-org -> ocean-origin/PORT-10933-bug-snyk-ingesting-vulnerabilities-from-wrong-org 638: * [new branch] PORT-11183-fix-starlette-denial-of-service-do-s-via-multipart-form-data -> ocean-origin/PORT-11183-fix-starlette-denial-of-service-do-s-via-multipart-form-data 639: * [new branch] PORT-11220-automate-git-tag-when-pushing-a-new-commit-to-main -> ocean-origin/PORT-11220-automate-git-tag-when-pushing-a-new-commit-to-main 640: * [new branch] PORT-11682-ocean-handle-and-log-errors-of-client-timeouts-when-interacting-with-port-api-upserting-etc -> ocean-origin/PORT-11682-ocean-handle-and-log-errors-of-client-timeouts-when-interacting-with-port-api-upserting-etc 641: * [new branch] PORT-12105-Bug-Pagination-logic-error-in-the-SonarQube-integration -> ocean-origin/PORT-12105-Bug-Pagination-logic-error-in-the-SonarQube-integration 642: * [new branch] PORT-12472/gitlab-merge-request-event -> ocean-origin/PORT-12472/gitlab-merge-request-event ... 665: * [new branch] PORT-14515 -> ocean-origin/PORT-14515 666: * [new branch] PORT-14517 -> ocean-origin/PORT-14517 667: * [new branch] PORT-14520 -> ocean-origin/PORT-14520 668: * [new branch] PORT-14521 -> ocean-origin/PORT-14521 669: * [new branch] PORT-14554-Prepare-A-Script-To-Remove-Dangling-Entities -> ocean-origin/PORT-14554-Prepare-A-Script-To-Remove-Dangling-Entities 670: * [new branch] PORT-14640-shutting-down-gracefully-on-ocean-resync-exception -> ocean-origin/PORT-14640-shutting-down-gracefully-on-ocean-resync-exception 671: * [new branch] PORT-14823 -> ocean-origin/PORT-14823 672: * [new branch] PORT-14828 -> ocean-origin/PORT-14828 673: * [new branch] PORT-14837 -> ocean-origin/PORT-14837 674: * [new branch] PORT-4308-ocean-support-pydantic-v-2 -> ocean-origin/PORT-4308-ocean-support-pydantic-v-2 675: * [new branch] PORT-8504-add-configurable-limit-for-wiz-import-issues -> ocean-origin/PORT-8504-add-configurable-limit-for-wiz-import-issues 676: * [new branch] PORT-8596-add-the-ability-to-ingest-ami-images-and-acm-certificate-from-aws-new-integration-into-port -> ocean-origin/PORT-8596-add-the-ability-to-ingest-ami-images-and-acm-certificate-from-aws-new-integration-into-port 677: * [new branch] PORT-8788-Add-deployment-method-requirements-field -> ocean-origin/PORT-8788-Add-deployment-method-requirements-field 678: * [new branch] PORT-9072 -> ocean-origin/PORT-9072 679: * [new branch] PORT-9097-ocean-icons-are-not-synced -> ocean-origin/PORT-9097-ocean-icons-are-not-synced 680: * [new branch] PORT-9326-bug-sonarqube-integration-throws-errors -> ocean-origin/PORT-9326-bug-sonarqube-integration-throws-errors 681: * [new branch] PORT-9612-add-sonar-project-properties-file-to-ocean-integration-code-base -> ocean-origin/PORT-9612-add-sonar-project-properties-file-to-ocean-integration-code-base ... 746: * [new branch] feat-gitlabv2-dependencies-and-vulnerabilities -> ocean-origin/feat-gitlabv2-dependencies-and-vulnerabilities 747: * [new branch] feat-remove-polling-logs -> ocean-origin/feat-remove-polling-logs 748: * [new branch] feat/bitbucket -> ocean-origin/feat/bitbucket 749: * [new branch] feat/port-7767-support-exporting-jira-issues-by-sprint -> ocean-origin/feat/port-7767-support-exporting-jira-issues-by-sprint 750: * [new branch] feature-bump-single-integration -> ocean-origin/feature-bump-single-integration 751: * [new branch] fetch_and_jq -> ocean-origin/fetch_and_jq 752: * [new branch] fix-dev-komodor-integration -> ocean-origin/fix-dev-komodor-integration 753: * [new branch] fix-memory-leak-with-fastapi -> ocean-origin/fix-memory-leak-with-fastapi 754: * [new branch] fix-unexpected-lack-of-config -> ocean-origin/fix-unexpected-lack-of-config 755: * [new branch] fix/strip-app-host -> ocean-origin/fix/strip-app-host 756: * [new branch] format_url_types -> ocean-origin/format_url_types 757: * [new branch] gitlab-param-change -> ocean-origin/gitlab-param-change 758: * [new branch] gitlab-v2 -> ocean-origin/gitlab-v2 759: * [new branch] jira-oauth-support -> ocean-origin/jira-oauth-support 760: * [new branch] jortegac/main -> ocean-origin/jortegac/main 761: * [new branch] jq_next-assertion-error -> ocean-origin/jq_next-assertion-error 762: * [new branch] label-prs -> ocean-origin/label-prs 763: * [new branch] main -> ocean-origin/main 764: * [new branch] mapping_rule_schema_fix -> ocean-origin/mapping_rule_schema_fix 765: * [new branch] new_base_image -> ocean-origin/new_base_image 766: * [new branch] oauth-token-refresh-by-error-mode -> ocean-origin/oauth-token-refresh-by-error-mode 767: * [new branch] ocean_11_memo_leak -> ocean-origin/ocean_11_memo_leak ... 828: * [new branch] snyk-upgrade-bfd8d053c19439c4e38f45827827ea7e -> ocean-origin/snyk-upgrade-bfd8d053c19439c4e38f45827827ea7e 829: * [new branch] snyk-upgrade-c0cc22d8bc9a14f9f9b16f4f2601f29d -> ocean-origin/snyk-upgrade-c0cc22d8bc9a14f9f9b16f4f2601f29d 830: * [new branch] snyk-upgrade-c360637938d20f9dc0bfa4b8e6e9c425 -> ocean-origin/snyk-upgrade-c360637938d20f9dc0bfa4b8e6e9c425 831: * [new branch] snyk-upgrade-c4308db9fedbf20f486a386e785bcec5 -> ocean-origin/snyk-upgrade-c4308db9fedbf20f486a386e785bcec5 832: * [new branch] snyk-upgrade-d18f88847cd5952ae3466151a60ac401 -> ocean-origin/snyk-upgrade-d18f88847cd5952ae3466151a60ac401 833: * [new branch] snyk-upgrade-d3b08700f6c7b35ad696d6e0ccbb7c19 -> ocean-origin/snyk-upgrade-d3b08700f6c7b35ad696d6e0ccbb7c19 834: * [new branch] snyk-upgrade-e030c59e680e225c2560aa5c7c7fb864 -> ocean-origin/snyk-upgrade-e030c59e680e225c2560aa5c7c7fb864 835: * [new branch] snyk-upgrade-ec34358e4330c12b9936c44978129497 -> ocean-origin/snyk-upgrade-ec34358e4330c12b9936c44978129497 836: * [new branch] snyk-upgrade-ef6788d7d595933a67b7b289999f37ee -> ocean-origin/snyk-upgrade-ef6788d7d595933a67b7b289999f37ee 837: * [new branch] snyk-upgrade-ff035b88aaaed39de6c7f6b33c85f98d -> ocean-origin/snyk-upgrade-ff035b88aaaed39de6c7f6b33c85f98d 838: * [new branch] test-fastapi -> ocean-origin/test-fastapi 839: * [new branch] test-pr -> ocean-origin/test-pr 840: * [new branch] update-depdendencies -> ocean-origin/update-depdendencies 841: * [new branch] update_docs -> ocean-origin/update_docs 842: * [new branch] upserts_no_save -> ocean-origin/upserts_no_save 843: ERROR: Ocean version in integrations/gitlab-v2/pyproject.toml is not updated to latest version -> 0.24.2 844: ##[error]Process completed with exit code 1. 845: Post job cleanup.

[qodo-merge-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Error Handling The get_paginated_project_resource method catches all exceptions and yields an empty list. This broad exception handling might mask unexpected errors that should be properly propagated. except Exception as e: logger.warning( f"Failed to fetch {resource_type} for project {project_path}: {str(e)}" ) yield [] Potential Bug The get_file_data method now conditionally decodes content only if response exists, but doesn't check if the 'content' key exists in the response before attempting to decode it. if response: response["content"] = base64.b64decode(response["content"]).decode("utf-8")

[qodo-merge-pro]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Improve type handling The code assumes response is either a list or should be converted to an empty list, but doesn't handle the case where response could be None or another unexpected type. This could lead to runtime errors. integrations/gitlab-v2/gitlab/clients/rest_client.py [108-110] response = await self.send_api_request("GET", path, params=params) -# HTTP API returns a list directly, or empty dict for 404 +# HTTP API returns a list directly, or empty dict/None for errors batch: list[dict[str, Any]] = response if isinstance(response, list) else [] [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 2 __ Why: The improved_code is essentially identical to the existing_code except for a minor comment change, offering minimal improvement to the actual functionality.	Low
Update

[shariff-6]

LGTM

[mk-armah]

How do we handle logs when a CI/CD does not exist and we want to skip ?