Sync upstream master

.github/pull_request_template.md

@@ -1,24 +1,63 @@
+### Type of change
+<!-- Mark the relevant option with an "x" -->
+- [ ] Bug fix (non-breaking change that fixes an issue)
+- [ ] New feature (non-breaking change that adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to change)
+- [ ] Refactoring (no functional changes)
+- [ ] Documentation update
+- [ ] Other (please describe):
+
+
 ### Summary
-> Describe your changes.
+<!-- Describe WHAT your changes do and WHY they are needed. -->
 
 
 
 ### Related issues or links
-> Include links to relevant issues or other pages.
+<!-- Include links to relevant issues or other pages. Use "Fixes #123" or "Closes #123" to auto-close issues. -->
+
+- Fixes #
+
+
+### Breaking changes
+<!-- If this PR introduces breaking changes, describe the impact and migration path. Otherwise, delete this section. -->
+
+
+
+### How was this tested?
+<!-- Describe how you tested your changes. Include relevant details such as test configuration, commands run, or manual testing steps. -->
 
-- https://github.com/cartography-cncf/cartography/issues/...
 
 
 ### Checklist
 
-Provide proof that this works (this makes reviews move faster). Please perform one or more of the following:
-- [ ] Update/add unit or integration tests.
-- [ ] Include a screenshot showing what the graph looked like before and after your changes.
-- [ ] Include console log trace showing what happened before and after your changes.
+#### General
+- [ ] I have read the [contributing guidelines](https://cartography-cncf.github.io/cartography/dev/developer-guide.html).
+- [ ] The linter passes locally (`make lint`).
+- [ ] I have added/updated tests that prove my fix is effective or my feature works.
+
+#### Proof of functionality
+<!-- Provide at least one of the following to help reviewers verify your changes: -->
+- [ ] Screenshot showing the graph before and after changes.
+- [ ] New or updated unit/integration tests.
+
+#### If you are adding or modifying a synced entity
+- [ ] Included Cartography sync logs from a real environment demonstrating successful synchronization of the new/modified entity. Logs should show:
+  - The sync job starting and completing without errors
+  - The number of nodes/relationships created or updated
+  - Example:
+    ```
+    INFO:cartography.intel.aws.ec2:Loading 42 EC2 instances for region us-east-1
+    INFO:cartography.intel.aws.ec2:Synced EC2 instances in 3.21 seconds
+    ```
+
+#### If you are changing a node or relationship
+- [ ] Updated the [schema documentation](https://github.com/cartography-cncf/cartography/tree/master/docs/root/modules).
+- [ ] Updated the [schema README](https://github.com/cartography-cncf/cartography/blob/master/docs/schema/README.md).
+
+#### If you are implementing a new intel module
+- [ ] Used the NodeSchema [data model](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#defining-a-node).
 
-If you are changing a node or relationship:
-- [ ] Update the [schema](https://github.com/cartography-cncf/cartography/tree/master/docs/root/modules) and [readme](https://github.com/cartography-cncf/cartography/blob/master/docs/schema/README.md).
 
-If you are implementing a new intel module:
-- [ ] Use the NodeSchema [data model](https://cartography-cncf.github.io/cartography/dev/writing-intel-modules.html#defining-a-node).
-- [ ] Confirm that the linter actually passes (submitting a PR where the linter fails shows reviewers that you did not test your code and will delay your review).
+### Notes for reviewers
+<!-- Optional: Add any context that would help reviewers, such as areas to focus on, design decisions, or open questions. -->

.github/workflows/ossf-scorecard.yml

@@ -57,6 +57,6 @@ jobs:
 
       # Upload the results to GitHub's code scanning dashboard.
       - name: "Upload to code-scanning"
-        uses: github/codeql-action/upload-sarif@1b168cd39490f61582a9beae412bb7057a6b2c4e # v3.29.5
+        uses: github/codeql-action/upload-sarif@cdefb33c0f6224e58673d9004f47f7cb3e328b89 # v3.29.5
         with:
           sarif_file: results.sarif

.github/workflows/publish-to-ghcr-and-pypi.yml

@@ -19,7 +19,7 @@ jobs:
         with:
           python-version: "3.10"
       - name: Install uv
-        uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867 # v7.1.6
+        uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
         with:
           enable-cache: true
           cache-dependency-glob: "uv.lock"
@@ -42,7 +42,7 @@ jobs:
     steps:
       # See: https://docs.docker.com/build/ci/github-actions/multi-platform/#build-and-load-multi-platform-images
       - name: Set up Docker
-        uses: docker/setup-docker-action@e61617a16c407a86262fb923c35a616ddbe070b3 # v4.6.0
+        uses: docker/setup-docker-action@e43656e248c0bd0647d3f5c195d116aacf6fcaf4 # v4.7.0
         with:
           daemon-config: |
             {
@@ -63,7 +63,7 @@ jobs:
         uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3.7.0
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
+        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
 
       - name: Login to GitHub Container Registry
         uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0

.github/workflows/sphinx.yml

@@ -31,7 +31,7 @@ jobs:
         with:
           python-version: "3.10"
       - name: Install uv
-        uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867 # v7.1.6
+        uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
         with:
           enable-cache: true
           cache-dependency-glob: "uv.lock"

.github/workflows/test_suite.yml

@@ -19,7 +19,7 @@ jobs:
         with:
           python-version: "3.10"
       - name: Install uv
-        uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244 # v7.1.4
+        uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
       - name: Check lockfile is up-to-date
         run: uv lock --check
       - uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
@@ -44,7 +44,7 @@ jobs:
         with:
           python-version: "3.10"
       - name: Install uv
-        uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867 # v7.1.6
+        uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
         with:
           enable-cache: true
           cache-dependency-glob: "uv.lock"
@@ -76,7 +76,7 @@ jobs:
         with:
           python-version: "3.10"
       - name: Install uv
-        uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867 # v7.1.6
+        uses: astral-sh/setup-uv@61cb8a9741eeb8a550a1b8544337180c0fc8476b # v7.2.0
         with:
           enable-cache: true
           cache-dependency-glob: "uv.lock"
@@ -102,7 +102,7 @@ jobs:
           images: ghcr.io/${{ github.repository }}
 
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
+        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
 
       - name: Build
         uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0

.gitignore

@@ -22,7 +22,6 @@ cartography/_version.py
 poetry.lock
 .cursor-server
 .gitconfig
-.python-version
 .python_history
 github_config.json
 .env.local

.python-version

@@ -0,0 +1 @@
+3.10

AGENTS.md

@@ -19,6 +19,32 @@ This guide teaches you how to write intel modules for Cartography using the mode
 - `cartography/graph/job.py` - Cleanup job utilities
 - `cartography/models/core/` - Base data model classes
 
+## 📝 Git and Pull Request Guidelines
+
+**Signing Commits**: All commits must be signed using the `-s` flag. This adds a `Signed-off-by` line to your commit message, certifying that you have the right to submit the code under the project's license.
+
+```bash
+# Sign a commit with a message
+git commit -s -m "feat(module): add new feature"
+```
+
+**Pull Request Descriptions**: When creating a pull request, use the template at `.github/pull_request_template.md`.
+
+Example PR creation:
+```bash
+gh pr create --title "feat(core): add BufferError retry handling" --body "$(cat <<'EOF'
+### Summary
+Add retry handling for BufferError to cartography's core Neo4j retry logic.
+
+### Related issues or links
+- https://github.com/cartography-cncf/cartography/issues/1234
+
+### Checklist
+- [x] Update/add unit or integration tests.
+EOF
+)"
+```
+
 ## 📋 Table of Contents
 
 1. @Quick Start: Copy an Existing Module

Dockerfile

@@ -1,5 +1,5 @@
 # Base image
-FROM python:3.10.19-slim@sha256:fb1feae978f1729094eb0405e5f9564e55b2b3b24db3261d30ba4f22c5001a8a AS base
+FROM python:3.10.19-slim@sha256:f5d029fe39146b08200bcc73595795ac19b85997ad0e5001a02c7c32e8769efa AS base
 # Default to ''. Overridden with a specific version specifier e.g. '==0.98.0' by build args or from GitHub actions.
 ARG VERSION_SPECIFIER
 # the UID and GID to run cartography as

README.md

@@ -22,22 +22,22 @@ You can learn more about the story behind Cartography in our [presentation at BS
 
 ## Supported platforms
 - [Airbyte](https://cartography-cncf.github.io/cartography/modules/airbyte/index.html) - Organization, Workspace, User, Source, Destination, Connection, Tag, Stream
-- [Amazon Web Services](https://cartography-cncf.github.io/cartography/modules/aws/index.html) - ACM, API Gateway, CloudWatch, CodeBuild, Config, Cognito, EC2, ECS, ECR (including multi-arch images, image layers, and attestations), EFS, Elasticsearch, Elastic Kubernetes Service (EKS), DynamoDB, Glue,  GuardDuty, IAM, Inspector, KMS, Lambda, RDS, Redshift, Route53, S3, Secrets Manager(Secret Versions), Security Hub, SNS, SQS, SSM, STS, Tags
+- [Amazon Web Services](https://cartography-cncf.github.io/cartography/modules/aws/index.html) - ACM, API Gateway, Bedrock, CloudWatch, CodeBuild, Config, Cognito, EC2, ECS, ECR (including multi-arch images, image layers, and attestations), EFS, Elasticsearch, Elastic Kubernetes Service (EKS), DynamoDB, Glue,  GuardDuty, IAM, Inspector, KMS, Lambda, RDS, Redshift, Route53, S3, SageMaker, Secrets Manager(Secret Versions), Security Hub, SNS, SQS, SSM, STS, Tags
 - [Anthropic](https://cartography-cncf.github.io/cartography/modules/anthropic/index.html) - Organization, ApiKey, User, Workspace
 - [BigFix](https://cartography-cncf.github.io/cartography/modules/bigfix/index.html) - Computers
 - [Cloudflare](https://cartography-cncf.github.io/cartography/modules/cloudflare/index.html) - Account, Role, Member, Zone, DNSRecord
 - [Crowdstrike Falcon](https://cartography-cncf.github.io/cartography/modules/crowdstrike/index.html) - Hosts, Spotlight vulnerabilities, CVEs
 - [DigitalOcean](https://cartography-cncf.github.io/cartography/modules/digitalocean/index.html)
 - [Duo](https://cartography-cncf.github.io/cartography/modules/duo/index.html) - Users, Groups, Endpoints
 - [GitHub](https://cartography-cncf.github.io/cartography/modules/github/index.html) - repos, branches, users, teams, dependency graph manifests, dependencies
-- [Google Cloud Platform](https://cartography-cncf.github.io/cartography/modules/gcp/index.html) - Bigtable, Cloud Resource Manager, Compute, DNS, Storage, Google Kubernetes Engine
+- [Google Cloud Platform](https://cartography-cncf.github.io/cartography/modules/gcp/index.html) - Bigtable, Cloud Functions, Cloud Resource Manager, Cloud Run, Cloud SQL, Compute, DNS, IAM, KMS, Secret Manager, Storage, Google Kubernetes Engine, Vertex AI
 - [Google GSuite](https://cartography-cncf.github.io/cartography/modules/gsuite/index.html) - users, groups (deprecated - use Google Workspace instead)
 - [Google Workspace](https://cartography-cncf.github.io/cartography/modules/googleworkspace/index.html) - users, groups, devices, OAuth apps
 - [Kandji](https://cartography-cncf.github.io/cartography/modules/kandji/index.html) - Devices
 - [Keycloak](https://cartography-cncf.github.io/cartography/modules/keycloak/index.html) - Realms, Users, Groups, Roles, Scopes, Clients, IdentityProviders, Authentication Flows, Authentication Executions, Organizations, Organization Domains
 - [Kubernetes](https://cartography-cncf.github.io/cartography/modules/kubernetes/index.html) - Cluster, Namespace, Service, Pod, Container, ServiceAccount, Role, RoleBinding, ClusterRole, ClusterRoleBinding, OIDCProvider
 - [Lastpass](https://cartography-cncf.github.io/cartography/modules/lastpass/index.html) - users
-- [Microsoft Azure](https://cartography-cncf.github.io/cartography/modules/azure/index.html) - App Service, Container Instance, CosmosDB, Data Factory, Event Grid, Functions, Azure Kubernetes Service (AKS), Load Balancer, Logic Apps, Resource Group, SQL, Storage, Virtual Machine, Virtual Networks
+- [Microsoft Azure](https://cartography-cncf.github.io/cartography/modules/azure/index.html) - App Service, Container Instance, CosmosDB, Data Factory, Event Grid, Functions, Key Vault, Azure Kubernetes Service (AKS), Load Balancer, Logic Apps, Resource Group, SQL, Storage, Virtual Machine, Virtual Networks
 - [Microsoft Entra ID](https://cartography-cncf.github.io/cartography/modules/entra/index.html) -  Users, Groups, Applications, OUs, App Roles, federation to AWS Identity Center
 - [NIST CVE](https://cartography-cncf.github.io/cartography/modules/cve/index.html) - Common Vulnerabilities and Exposures (CVE) data from NIST database
 - [Okta](https://cartography-cncf.github.io/cartography/modules/okta/index.html) - users, groups, organizations, roles, applications, factors, trusted origins, reply URIs, federation to AWS roles, federation to AWS Identity Center
@@ -142,6 +142,7 @@ Get started with our [developer documentation](https://cartography-cncf.github.i
 1. [Cloudanix](https://www.cloudanix.com/)
 1. [Corelight](https://www.corelight.com/)
 1. [SubImage](https://subimage.io)
+1. [Superhuman](https://superhuman.com/)
 1. {Your company here} :-)
 
 If your organization uses Cartography, please file a PR and update this list. Say hi on Slack too!

cartography/cli.py

@@ -620,6 +620,60 @@ def _build_parser(self):
             default=None,
             help=("The Duo api hostname"),
         )
+        parser.add_argument(
+            "--gitlab-url",
+            type=str,
+            default="https://gitlab.com",
+            help=(
+                "The GitLab instance URL. Defaults to https://gitlab.com. "
+                "Set to your self-hosted instance URL if applicable (e.g., https://gitlab.example.com)."
+            ),
+        )
+        parser.add_argument(
+            "--gitlab-token-env-var",
+            type=str,
+            default=None,
+            help=(
+                "The name of environment variable containing the GitLab personal access token. "
+                "Required if you are using the GitLab intel module. Ignored otherwise."
+            ),
+        )
+        parser.add_argument(
+            "--gitlab-organization-id",
+            type=int,
+            default=None,
+            help=(
+                "The GitLab organization (top-level group) ID to sync. "
+                "Required if you are using the GitLab intel module. Ignored otherwise."
+            ),
+        )
+        parser.add_argument(
+            "--workday-api-url",
+            type=str,
+            default=None,
+            help=(
+                "The Workday API URL. "
+                "Required if you are using the Workday intel module. Ignored otherwise."
+            ),
+        )
+        parser.add_argument(
+            "--workday-api-login",
+            type=str,
+            default=None,
+            help=(
+                "The Workday API login username. "
+                "Required if you are using the Workday intel module. Ignored otherwise."
+            ),
+        )
+        parser.add_argument(
+            "--workday-api-password-env-var",
+            type=str,
+            default=None,
+            help=(
+                "The name of environment variable containing the Workday API password. "
+                "Required if you are using the Workday intel module. Ignored otherwise."
+            ),
+        )
         parser.add_argument(
             "--semgrep-app-token-env-var",
             type=str,
@@ -1224,6 +1278,31 @@ def main(self, argv: str) -> int:
             config.duo_api_key = None
             config.duo_api_secret = None
 
+        # GitLab config
+        if config.gitlab_url and config.gitlab_token_env_var:
+            logger.debug(
+                f"Reading GitLab token from environment variable {config.gitlab_token_env_var}",
+            )
+            config.gitlab_token = os.environ.get(config.gitlab_token_env_var)
+        else:
+            config.gitlab_token = None
+        # gitlab_organization_id is passed directly from CLI arg (type=int)
+
+        # Workday config
+        if (
+            config.workday_api_url
+            and config.workday_api_login
+            and config.workday_api_password_env_var
+        ):
+            logger.debug(
+                f"Reading Workday API password from environment variable {config.workday_api_password_env_var}",
+            )
+            config.workday_api_password = os.environ.get(
+                config.workday_api_password_env_var
+            )
+        else:
+            config.workday_api_password = None
+
         # Semgrep config
         if config.semgrep_app_token_env_var:
             logger.debug(
@@ -1438,6 +1517,7 @@ def main(argv=None):
     logging.getLogger("neo4j").setLevel(logging.WARNING)
     logging.getLogger("azure.identity").setLevel(logging.WARNING)
     logging.getLogger("httpx").setLevel(logging.WARNING)
+    logging.getLogger("slack_sdk").setLevel(logging.WARNING)
     logging.getLogger("azure.core.pipeline.policies.http_logging_policy").setLevel(
         logging.WARNING
     )