--tf-exclude-downloaded-modules not working #5408

bkonicek-calm · 2023-10-19T14:10:10Z

bkonicek-calm
Oct 19, 2023

Description

I have a Terraform config that calls several of our custom modules. After running a terraform plan, they are downloaded to .terraform/modules. Trivy correctly identifies misconfigurations in them. However, when I add the --tf-exclude-downloaded-modules option, Trivy still checks those modules and reports misconfigurations.

Desired Behavior

Trivy should not identify misconfigurations in downloaded modules when the --tf-exclude-downloaded-modules option is set

Actual Behavior

Trivy scans and reports on misconfigurations in downloaded modules

Reproduction Steps

1. Have a Terraform config that references modules hosted in a git repo. E.g. `"git::https://github.com/ORG/REPO.git//gcp-modules/gke_cluster?ref=gke_cluster-0.0.7"`
2. Run `terraform init`, `terraform plan`
3. Run `trivy config . --debug --tf-exclude-downloaded-modules`

Target

Filesystem

Scanner

Misconfiguration

Output Format

Table

Mode

Standalone

Debug Output

❯ trivy config . --debug --severity CRITICAL --tf-exclude-downloaded-modules
2023-10-19T10:04:38.966-0400    DEBUG   Severities: ["CRITICAL"]
2023-10-19T10:04:38.980-0400    DEBUG   cache dir:  /Users/bkonicek/Library/Caches/trivy
2023-10-19T10:04:38.980-0400    INFO    Misconfiguration scanning is enabled
2023-10-19T10:04:38.981-0400    DEBUG   Policies successfully loaded from disk
2023-10-19T10:04:39.007-0400    DEBUG   The nuget packages directory couldn't be found. License search disabled
2023-10-19T10:04:39.008-0400    DEBUG   Walk the file tree rooted at '.' in parallel
2023-10-19T10:04:39.012-0400    DEBUG   Scanning Helm files for misconfigurations...
2023-10-19T10:04:39.082-0400    DEBUG   Scanning Terraform files for misconfigurations...
2023-10-19T10:04:49.887-0400    DEBUG   Scanning Terraform Plan files for misconfigurations...
2023-10-19T10:04:50.286-0400    DEBUG   OS is not detected.
2023-10-19T10:04:50.286-0400    INFO    Detected config files: 86
2023-10-19T10:04:50.286-0400    DEBUG   Scanned config file: 
2023-10-19T10:04:50.286-0400    DEBUG   Scanned config file: .terraform/modules/filemage/external-services-modules/datadog_monitor
2023-10-19T10:04:50.286-0400    DEBUG   Scanned config file: .terraform/modules/gke_cluster/external-services-modules/datadog_monitor
2023-10-19T10:04:50.286-0400    DEBUG   Scanned config file: .terraform/modules/gke_cluster/gcp-modules/bigquery
2023-10-19T10:04:50.286-0400    DEBUG   Scanned config file: .terraform/modules/gke_cluster/gcp-modules/bigquery/main.tf
2023-10-19T10:04:50.286-0400    DEBUG   Scanned config file: 
...
...

Operating System

macOS Ventura

Version

❯ trivy --version
Version: 0.46.0

Checklist

Run trivy image --reset
Read the troubleshooting

simar7 · 2023-10-20T04:09:08Z

simar7
Oct 20, 2023
Maintainer

I'm not able to reproduce this. You can see an example here https://gist.github.com/simar7/b02a16bb683adfa147c72159d727722c - In the non-excluded file you can see a variety of misconfiguration issues that are from downloaded modules.

Can you share your terraform config file so we can run locally to try it out?

2 replies

bkonicek-calm Oct 20, 2023
Author

Here is a quick example

# main.tf
resource "google_compute_firewall" "app" {
  project       = "foo"
  name          = "test"
  network       = "foo"
  priority      = "1001"
  source_ranges = ["0.0.0.0/0"]

  allow {
    protocol = "tcp"
    ports    = ["443", "80"]
  }

  target_tags = ["foo"]
}

module "vm" {
  source = "git::https://github.com/terraform-google-modules/terraform-google-vm.git?ref=v10.1.0"
}

terraform init
trivy config . --tf-exclude-downloaded-modules --debug outputs

❯ trivy config . --tf-exclude-downloaded-modules --debug
2023-10-20T09:36:30.692-0400    DEBUG   Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2023-10-20T09:36:30.702-0400    DEBUG   cache dir:  /Users/benkonicek/Library/Caches/trivy
2023-10-20T09:36:30.702-0400    INFO    Misconfiguration scanning is enabled
2023-10-20T09:36:30.703-0400    DEBUG   Policies successfully loaded from disk
2023-10-20T09:36:30.711-0400    DEBUG   The nuget packages directory couldn't be found. License search disabled
2023-10-20T09:36:30.729-0400    DEBUG   Walk the file tree rooted at '.' in parallel
2023-10-20T09:36:30.734-0400    DEBUG   Scanning Helm files for misconfigurations...
2023-10-20T09:36:30.742-0400    DEBUG   Scanning Kubernetes files for misconfigurations...
2023-10-20T09:36:32.857-0400    DEBUG   Scanning Terraform files for misconfigurations...
2023-10-20T09:36:33.812-0400    DEBUG   OS is not detected.
2023-10-20T09:36:33.812-0400    INFO    Detected config files: 39
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/instance_template/encrypted_disks
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/instance_template/encrypted_disks/main.tf
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/mig/full
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/mig/healthcheck/main.tf
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/mig_stateful/main.tf
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/umig/full
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/compute_instance/multiple_interfaces
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/compute_instance/multiple_interfaces/main.tf
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/compute_instance/next_hop
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/compute_instance/next_hop/main.tf
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/compute_instance/tags
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/examples/compute_instance/tags/main.tf
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/metadata.yaml
2023-10-20T09:36:33.812-0400    DEBUG   Scanned config file: .terraform/modules/vm/modules/preemptible_and_regular_instance_templates/metadata.yaml
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/modules/umig/metadata.yaml
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/modules/compute_disk_snapshot/metadata.yaml
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/modules/compute_instance/metadata.yaml
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/modules/instance_template/metadata.yaml
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/modules/mig/metadata.yaml
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/modules/mig_with_percent/metadata.yaml
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/compute_instance/simple_zone
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/compute_instance/disk_snapshot
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/instance_simple
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/mig/autoscaler
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/mig/healthcheck
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/mig/simple
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/mig_with_percent/simple
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/shared
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/shared/network.tf
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/instance_template/additional_disks
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/instance_template/alias_ip_range
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/instance_template/simple
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/mig_stateful
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/preemptible_and_regular_instance_templates/simple
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/umig/named_ports
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/umig/simple
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/fixtures/umig/static_ips
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/setup
2023-10-20T09:36:33.813-0400    DEBUG   Scanned config file: .terraform/modules/vm/test/setup/iam.tf

.terraform/modules/vm/examples/compute_instance/multiple_interfaces/main.tf (terraform)

Tests: 12 (SUCCESSES: 7, FAILURES: 5, EXCEPTIONS: 0)
Failures: 5 (UNKNOWN: 0, LOW: 1, MEDIUM: 4, HIGH: 0, CRITICAL: 0)

MEDIUM: Instance allows use of project-level SSH keys.
════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Use of project-wide SSH keys means that a compromise of any one of these key pairs can result in all instances being compromised. It is recommended to use instance-level keys.

See https://avd.aquasec.com/misconfig/avd-gcp-0030
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 .terraform/modules/vm/examples/compute_instance/multiple_interfaces/main.tf:18-36
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
  18 ┌ resource "google_compute_instance" "default" {
  19 │   project      = var.project_id # Replace with your project ID in quotes
  20 │   zone         = "us-central1-b"
  21 │   name         = "backend-instance"
  22 │   machine_type = "e2-medium"
  23 │   boot_disk {
  24 │     initialize_params {
  25 │       image = "debian-cloud/debian-9"
  26 └     }
  ..   
...

Additionally, as I mentioned in this discussion, when I have the "vm" module in my config, trivy does NOT see the misconfiguration for google_compute_firewall.app allowing 0.0.0.0/0.

❯ trivy config . --tf-exclude-downloaded-modules -s CRITICAL
2023-10-20T09:39:51.985-0400    INFO    Misconfiguration scanning is enabled
2023-10-20T09:39:55.134-0400    INFO    Detected config files: 39

However, if I delete my .terraform folder, I get

❯ trivy config . --tf-exclude-downloaded-modules -s CRITICAL
2023-10-20T09:40:07.208-0400    INFO    Misconfiguration scanning is enabled
2023-10-20T09:40:07.596-0400    INFO    Detected config files: 2

main.tf (terraform)

Tests: 1 (SUCCESSES: 0, FAILURES: 1, EXCEPTIONS: 0)
Failures: 1 (CRITICAL: 1)

CRITICAL: Firewall rule allows ingress traffic from multiple addresses on the public internet.
════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Network security rules should not use very broad subnets.

Where possible, segments should be broken into smaller subnets and avoid using the <code>/0</code> subnet.

See https://avd.aquasec.com/misconfig/avd-gcp-0027
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 main.tf:6
   via main.tf:1-14 (google_compute_firewall.app)
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   resource "google_compute_firewall" "app" {
   .   
   6 [   source_ranges = ["0.0.0.0/0"]
  ..   
  14   }

bkonicek-calm Oct 20, 2023
Author

If I add --skip-dirs .terraform as @nikpivkin suggested, then it does see my individual resource. It seems like when Trivy scans downloaded modules it's somehow causing it to skip standalone resources that are not part of a module within the same config as Scanned config file: main.tf does not appear in the debug logs of my previous example.

❯ terraform init
❯ trivy config . --tf-exclude-downloaded-modules -s CRITICAL --skip-dirs .terraform --debug
2023-10-20T09:44:18.675-0400    DEBUG   Severities: ["CRITICAL"]
2023-10-20T09:44:18.686-0400    DEBUG   cache dir:  /Users/benkonicek/Library/Caches/trivy
2023-10-20T09:44:18.686-0400    INFO    Misconfiguration scanning is enabled
2023-10-20T09:44:18.686-0400    DEBUG   Policies successfully loaded from disk
2023-10-20T09:44:18.691-0400    DEBUG   The nuget packages directory couldn't be found. License search disabled
2023-10-20T09:44:18.717-0400    DEBUG   Walk the file tree rooted at '.' in parallel
2023-10-20T09:44:18.717-0400    DEBUG   Skipping directory: .terraform
2023-10-20T09:44:18.717-0400    DEBUG   Scanning Terraform files for misconfigurations...
2023-10-20T09:44:19.076-0400    DEBUG   OS is not detected.
2023-10-20T09:44:19.076-0400    INFO    Detected config files: 2
2023-10-20T09:44:19.076-0400    DEBUG   Scanned config file: .
2023-10-20T09:44:19.076-0400    DEBUG   Scanned config file: main.tf

main.tf (terraform)

Tests: 1 (SUCCESSES: 0, FAILURES: 1, EXCEPTIONS: 0)
Failures: 1 (CRITICAL: 1)

CRITICAL: Firewall rule allows ingress traffic from multiple addresses on the public internet.
════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Network security rules should not use very broad subnets.

Where possible, segments should be broken into smaller subnets and avoid using the <code>/0</code> subnet.

See https://avd.aquasec.com/misconfig/avd-gcp-0027
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 main.tf:6
   via main.tf:1-14 (google_compute_firewall.app)
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   resource "google_compute_firewall" "app" {
   .   
   6 [   source_ranges = ["0.0.0.0/0"]
  ..   
  14   }

nikpivkin · 2023-10-20T11:31:28Z

nikpivkin
Oct 20, 2023
Maintainer

Hi @bkonicek-calm !

The --tf-exclude-downloaded-modules flag excludes the results of scanning downloaded modules specified in the source attribute. You can exclude the .terraform folder from scanning by using the skip-dirs flag: --skip-dirs ".terraform"

2 replies

bkonicek-calm Oct 20, 2023
Author

I guess I'm not fully understanding what the flag is supposed to do then, is it not meant to ignore .terraform?

trivy --help has this as the description for the flag

      --tf-exclude-downloaded-modules     remove results for downloaded modules in .terraform folder

nikpivkin Oct 20, 2023
Maintainer

@bkonicek-calm Thanks for finding it. Track #5416

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

--tf-exclude-downloaded-modules not working #5408

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 4 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

--tf-exclude-downloaded-modules not working #5408

bkonicek-calm Oct 19, 2023

Description

Desired Behavior

Actual Behavior

Reproduction Steps

Target

Scanner

Output Format

Mode

Debug Output

Operating System

Version

Checklist

Replies: 2 comments · 4 replies

simar7 Oct 20, 2023 Maintainer

bkonicek-calm Oct 20, 2023 Author

bkonicek-calm Oct 20, 2023 Author

nikpivkin Oct 20, 2023 Maintainer

bkonicek-calm Oct 20, 2023 Author

nikpivkin Oct 20, 2023 Maintainer

bkonicek-calm
Oct 19, 2023

Replies: 2 comments 4 replies

simar7
Oct 20, 2023
Maintainer

bkonicek-calm Oct 20, 2023
Author

bkonicek-calm Oct 20, 2023
Author

nikpivkin
Oct 20, 2023
Maintainer

bkonicek-calm Oct 20, 2023
Author

nikpivkin Oct 20, 2023
Maintainer