PR #2
Open
PR #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updated get_resource_name to handle when there's two 'name' values. Thank you to Sebastian Edge for pointing this out and for providing this fix.
* Add policy and examples for Private CA certificate lifetime Added documentation, Terraform examples, and OPA policy scaffolding for managing the lifetime of google_privateca_certificate_authority resources in GCP. This includes example configs, a policy template for lifetime enforcement, and supporting variable definitions. Also removed obsolete VSCode settings. * Pushing documentation * Add initial content to index.md * Add relative link to API Gateway documentation * Convert index.md to index.html with fetch script * Refactor script to dynamically fetch and display files * Update docs index.html to fetch GitHub files Refactored script to fetch and display files from GitHub repository directly in the HTML body. * Refactor HTML script to fetch repo contents correctly * Update API endpoint in index.html * Update fetch URL in index.html * Update API endpoint in index.html * Update title and heading for GCP documentation * Refactor script to improve folder rendering logic * Refactor HTML and JavaScript for GCP documentation * Add initial HTML structure for documentation * Update file links to include Policy-Deployment-Engine * Fix duplicate data fetching in index.html * Fix suburl construction for GitHub API fetch * Limit displayed items to 2 at each level * Refactor file fetching and limit display items * Modify display name and path handling for markdown files * Enhance index.html with styling and search feature * Update file fetching and display logic in index.html * Update index.html * Display all top-level folders instead of two Remove limit on top-level folder display * Change title and enhance search functionality Updated the title and added a warning message about API call limits. Adjusted styling and search functionality for better usability. * Moving script * New document generator * Rearranged script * Restore ReadME * Removing unnecessary index.html * Providing instructions for doc generation Deleted generated AlloyDB markdown documentation files and updated the resource_json for alloydb_backup to remove security/compliance metadata. Added a README to scripts explaining the documentation workflow, renamed markdown_builder.py to create_markdown.py, and doc_generator.py to original_document_generator.py for clarity. * Create subfolder for docgen scripts and move README
* Normalize security_impact as boolean in docgen Updated the docgen README and create_markdown.py to treat 'security_impact' as a boolean value instead of a string or null. Adjusted normalization logic and documentation to clarify expected types for 'required' and 'security_impact' fields. * Update docgen README with vertex_ai example Replaces references to 'alloydb' with 'vertex_ai' in the documentation and updates the example JSON resource to use 'vertex_ai_dataset' with more detailed argument documentation. This provides clearer guidance for documenting Vertex AI services.
* Security Policies Service - Model Armor * policies fix * Adding documentation
* Policy Create - service:cloud function gen2 * small revert * small revert * clean code * Security Policies * code clean * code clean * code clean * naming changes * naming changes * Non security related policy removal and location update * Non security related policy removal and location update * policies fix * Adding Documentation
* Identity_Aware_Proxy code import * identity_aware_proxy service's google_iap_brand resource changes * identity_aware_proxy changes and new attributes * IAP Changes * IAp Fixes * IAP Resource policy update * google_iap_setting.allowed_domain condition fixed * Identity_Aware_Proxy Documentation
* first commit * update the inputs folder logic
* gcp cloud service storage * updated asked changes * asked changes by yotam * naming cleanup * naming cleanup * default object access control resource * policies for some resources * add remaining policies * reviewed existing inherited policies * minor format * address pr comments * combine some policies * naming cleanup * various bug fixes and renaming for testing * merge commit * folders rename * terraform fmt * add documentation * folder rename and documentation added * doc generated * slight update to docs * slight update to docs --------- Co-authored-by: AkashreddyVootk <s224127166@deakin.edu.au>
* Restored deleted folders and applied review feedback * Addressed feedback and added documentation
* Work on cloud platform service * add inputs & policies of cloud_platform * Delete .vscode/settings.json * delete template comments from all policy files * undo changes to templates * delete comput and storage folders * update * fix: applied contributor feedback * Save local changes before merging * update to name for resource_value_name on vars.rego * remove exempt_members policy due to doesn't work and helper.rego hasn't update for this policy * Add/update JSON documentation for GCP resources - cloud platform * Add/update markdown documentation for GCP resources - cloud platform * all updated * readme.md back * Fix: resolved merge conflict in cloudfunctions2_function.json, removed markers and applied origin/dev security rules * Fix: resolved merge conflict in cloudfunctions2_function_iam.json, removed markers and applied origin/dev security rules * Fix: resolved merge conflict in model_armor_template.json & _floorsetting.json, removed markers and applied origin/dev security rules * google_project update name * update * update google_service_account_key/exposure * update google_folder_organization_policy/constraint
* finished script and basic pipeline * remove comment * finished script and basic pipeline * remove comment
* first commit * update the inputs folder logic * Fix: Root path error fix (Alloes to run linters from any path) but it must me called from the root directory "Policy-Deployment-Engine
* Add policies and inputs for Cloud Key Management * Updated helpers * updating * Updating the policies * final updates * Changes after the review * Initial changes * initial changs * vars repositioning * Changes in naming * final changes into, updating according to linters and pipeline * final commits * updated gitignore * added readme * update vars
…ation (#172) - Added 11 Looker Core security policies with proper OPA rules - Fixed duplicate variable declarations by creating separate variables.tf files - Included comprehensive documentation with security annotations - All policies pass terraform init/validate without errors - Using proper service branch structure as requested
Good catch sumedh
* Initial Commit * Updated privileged access manager entitlement policies * added changes * small change * remove policy
* removed selenium scrapping * remove cache json for service to use runtime cache * create readme * create readme
* Initial commit * Fixed review comments * Add all plan.json files * Pushing Lustre branch * Updated few files * Added documentation * Updated changes * Updated * Removed chronicle inputs and policy folder
* GKE Hub Feature (Fleet Observability) - require default logs routing mode COPY or MOVE (updated)
* Security policies for google_gke_hub_feature
* no_public_principals_binding policy for IAM binding
* no_public_principals_member policy for IAM member
* configmanagement security policies (git_secure_auth, git_approved_HTTPS, git_approved_dir, pc_enable_required)
* added policy_binding policy & authority_issuer policy
* GKEHub: remove deprecated google_gke_hub_feature_membership/git_approved_dir
* GKEHub: standardize resource_value_name to {c|nc}; fix authority_issuer; tighten RBAC/HTTPS controls
…186) * GKE Hub Policies (#107) * GKE Hub Feature (Fleet Observability) - require default logs routing mode COPY or MOVE (updated) * Security policies for google_gke_hub_feature * no_public_principals_binding policy for IAM binding * no_public_principals_member policy for IAM member * configmanagement security policies (git_secure_auth, git_approved_HTTPS, git_approved_dir, pc_enable_required) * added policy_binding policy & authority_issuer policy * GKEHub: remove deprecated google_gke_hub_feature_membership/git_approved_dir * GKEHub: standardize resource_value_name to {c|nc}; fix authority_issuer; tighten RBAC/HTTPS controls * Policies and documentation * helpers comment * Policies and terraform files from older branch * Required changes to helpers needed for policies * Fix wrong edit * Fixing naming for input files to be exact matches "c" or "nc" * Missed name from last commit, also formatting plan files * Changing name of folders to more descriptive label --------- Co-authored-by: Sebastian <sebastianedge50@gmail.com>
* Updated GDCE policies and input files for cluster security compliance * Updated GDCE policies, inputs, and documentation; added Cloud Stackdriver Logging service * "Updated Document of GDCE" * feat: add GDCE plan.json outputs only * Fixed GDCE policies: corrected node pool locations, KMS key validation, cluster target version, and maintenance policy checks. Removed CloudStackdriver logging service. * Remove Cloud Stackdriver Logging service from inputs and policies * added GDCE cluster CIDR blocks policy
* initial policy for dlp * temporary helpers.rego fix * polciies for network resource * polciies for network resource * policies for network peering resource * policies for network policy resource * policies for external access rule * merged helper rego * policy rewrite for new helper * fix policy * policy for external access rule * remove project from config * restructure and minor changes * folder cleanup * Add API Hub policies and inputs for peer review. * Checking policies and code * API Hub pull request * Changes made according to Pat Stewart PR * Added some more changes according to Pat Stuart PR * Markdown created for peer review * Peer review changes made for Api_Hub * Made two minor changes --------- Co-authored-by: Sumedh Vartak <67580847+sumedh004@users.noreply.github.com>
* Fixed Mistakes * Made Some Compliancy Changes * Made Some Requested Changes Made some Changes * Added Plan Files Added Plan Files * Pattern_whitelist Changed 2 policys to pattern whitelist * Changed_engine_schema_json PatternWhitelist * Updated_recommendation_engine Removed unneeded stuff. * Taking out Trash Gone (the location policys that arent in AUS) (also file adjustments for the engine_schema json check, back to whitelist but with full expression) * More Changes More Changes Made * Added location Policy's Back Yotam Said i could add them, because i did a few they will just be worth less. * misc some misc changes. * Documentation Hope this is what you wanted. * search_engine_fix * engine_config_fix * Opps Opps --------- Co-authored-by: Pat Stuart [SSW] <71543754+Pat-Stuart@users.noreply.github.com>
* Adjust policy check and add linter into workflow * adjust file name * removing linter from pipeline * Try plugin caching * Actually put the right name * Revert commands * try variant of commands * chnage google credentials * Disable backend * remove reconfigure * debug * modify * try this * clean up commands * change * debugging * debug * debug * debug * add cache env * add cleanup * try cleanup earlier * backend off * global cache * logging * add large folder logging * cleanup * abs path * find all tf * try this * cleanup
Recovered missing /templates folder with templates for c.tf, nc.tf, config.tf, vars.rego, policy.rego files This also allows folder_generator script to be functional
- Fixed the formatting issue that caused the wiki pages to display incorrectly. The tables were breaking due to missing newline characters after the headings. - Updated the create_markdown.py script to automatically insert the required newline - Re-ran the script to correct all existing Markdown files.
# Refactor Policy Helpers into Modular Rego Architecture
## Overview
Major refactor of the policy helper framework to improve
maintainability, testability, and extensibility. This introduces a
modular architecture with specialized policy modules, comprehensive test
coverage, and cloud-provider-agnostic design.
---
## What Changed
### **1. Modular Policy Architecture**
Broke down monolithic `policies/gcp/_helpers/helpers.rego` (590 lines)
into specialized modules:
```
policies/_helpers/
├── helpers.rego # Orchestration layer
├── shared.rego # Common utilities
└── policies/ # Policy-specific modules
├── blacklist.rego
├── whitelist.rego
├── range.rego
├── pattern_blacklist.rego
├── pattern_whitelist.rego
└── element_blacklist.rego
```
**Benefits:**
- Each policy type is self-contained and independently testable
- Easier to add new policy types without touching existing code
- Clear separation of concerns between orchestration and policy logic
### **2. Cloud-Provider Agnostic Design**
- Moved from `policies/gcp/_helpers/` → `policies/_helpers/`
- Changed namespace from `terraform.gcp.helpers` → `terraform.helpers`
- Can now be imported by AWS, Azure, and GCP policies
### **3. Comprehensive Test Suite**
Added 64 tests across 7 test files with multiple testing approaches:
**Test infrastructure:**
- `unit_test_helpers.sh` - Run all tests
- `smoke_test_helpers.sh` - Quick integration checks
- `policy_debug.sh` - Debug policy output
- `check_ux.sh` - Review violation messages
### **4. Enhanced Documentation**
- **`policies/_helpers/README.md`** (665 lines): Complete framework
documentation including architecture, usage guide, troubleshooting, and
migration notes
- **`tests/_helpers/README.md`** (175 lines): Test documentation with
fixture structure explanation
---
## Review Focus Areas
1. **Validation**: Are results equivalent to existing helpers?
---
## Next Steps
After merge:
1. Update existing GCP policies to use new namespace
(`terraform.helpers`) see Migration Impact below
2. Individual policies should be reviewed in detail
---
## Technical Improvements
### Standardized Interfaces
All policy modules now expose consistent interface:
```rego
# Evaluate a single condition against resources
evaluate_condition(condition, resources) := result
# Evaluate multiple conditions with AND logic
evaluate_and_conditions(conditions, resources) := result
```
### Shared Utilities (`shared.rego`)
Extracted common operations:
- `get_resource_attribute()` - Safe nested attribute access
- `format_path()` - Consistent path formatting
- `normalize_to_array()` - Handle single values or arrays
- `get_resources_by_type()` - Filter resources by type
---
## Migration Impact
**Existing policies** using `data.terraform.gcp.helpers` should be
updated to:
```rego
import data.terraform.helpers
```
**Example migration:**
```rego
# Before
package terraform.gcp.security.cloud_storage.google_storage_bucket.allowed_location
import data.terraform.gcp.helpers
# After
package terraform.gcp.security.cloud_storage.google_storage_bucket.allowed_location
import data.terraform.helpers
```
The old `policies/gcp/_helpers/helpers.rego` has been slimmed down (590
lines → minimal) but existing policies will continue to work during the
migration period.
---
## Validation Checklist
- [x] All 64 tests passing
- [x] Documentation complete with examples
- [x] No breaking changes to public API
- [x] OPA 1.2.0 compatible (Rego v1)
- [x] Cloud-provider agnostic design
- [x] Comprehensive test coverage (unit + integration)
Create the issue template for the new PDE Project board
…246) Get attribute value from a resource with null fallback Simplifies the common pattern of accessing nested resource attributes Fix: Array-of-Objects Field Extraction (Added 2025-12-04) When the attribute path ends with a string field name and leads to an array of objects, this function automatically extracts that field from each object in the array. Added corresponding tests
refactor: .gitignore organization and coverage - Add comprehensive IDE/editor ignores (VS Code, JetBrains) - Add OS-specific ignores (macOS, Windows, Linux) - Expand Python ignores (venv, cache, test reports) - Reorganize Terraform ignores with better documentation - Add clear section headers for maintainability"
Policies created for gemini
Completed the policy implementation for Google API Gateway resources. The following Rego policies were added - 1. google_api_gateway_api - labels 2. google_api_gateway_api_config - labels 3. IAM policies for API Config, API, and Gateway - *_iam_binding → members & role validation - *_iam_member → member & role validation - *_iam_policy → members & role validation
…253) Refactors how the auto-test script locates and loads shared policy helpers, ensuring robust and flexible handling of policy directory structures. The goal is to make it easier for users to run policy tests from service-specific directories while guaranteeing that the shared `_helpers` module is always available for OPA evaluation. Additionally, it removes a deprecated GCP helper shim, as all policies now use the unified helpers location. Key changes include: * Added a `normalize_policies_root` function in `auto_test.py` to traverse up from the user-provided policies directory to find the root containing the `_helpers` module. * Removed the obsolete `policies/gcp/_helpers/helpers.rego` shim, as GCP policies now directly use the unified `terraform.helpers` module. * Changed the `terraform show` command in `run_terraform_commands` to use a pipe (`| cat > plan.json`) for improved cross-platform compatibility.
Output of local run of auto-test - see my comment below for github
auto-test results explanation.
Summary of policy checks:
Service: access_context_manager_vpc_service_controls
Resource: google_access_context_manager_access_level
Policy: require_admin_approval - ✅
Policy: os_type - ✅
Policy: region - ✅
Policy: combining_function - ✅
Policy: require_corp_owned - ✅
Policy: allowed_encryption_statuses - ✅
Policy: require_screen_lock - ✅
Policy: allowed_device_management_levels - ✅
Resource: google_access_context_manager_access_level_condition
Policy: require_admin_approval - ✅
Policy: os_type - ✅
Policy: region - ✅
Policy: require_corp_owned - ✅
Policy: require_screen_lock - ✅
Resource: google_access_context_manager_access_levels
Policy: os_type - ✅
Policy: region - ✅
Policy: require_screen_lock - ✅
Resource: google_access_context_manager_service_perimeter
Policy: status - ✅
## Backup and Dr Service ready for merge ##
Doc generation version 2 for Terraform providers. See README.md for documentation. Note the branch does not include the generated output files.
Added BigQuery Analytics Hub security policies and generated service documentation. ## Policies implemented - Data Exchange: - Enforce approved location - Enforce allowed discovery_type - Data Exchange IAM Member: - Block public access (allUsers, allAuthenticatedUsers) - Data Exchange Subscription: - refresh_policy must not be NEVER - Listing: - restricted_export_config must be enabled - Listing IAM Member: - Block public access (allUsers, allAuthenticatedUsers) - Listing Subscription: - destination_dataset.labels.environment must be set (non-empty) ## Validation -OPA checks completed successfully - Auto tests passed in Git Bash - Documentation generated using scripts/docgen
Co-authored-by: Jake N <witm@wMacbookPro.local>
- Add 9 OPA security policies for google_vpc_access_connector resource - Policies cover: region, min_instances, max_instances, machine_type, ip_cidr_range, min_throughput, max_throughput, network, subnet - Include compliant and non-compliant Terraform test configurations - Enforce Australian data sovereignty (australia-southeast1/2) - Integrate with existing PDE helpers framework
GCP and Azure terraform provider markdown files escape underscores in resource titles (e.g., `# google\_biglake\_catalog`). The parser's regex matched the literal backslash, causing resource name extraction to fail or return malformed names. ### Changes - **Updated `extract_resource_name()` regex pattern**: Modified Pattern 2 from `_\S+` to `(?:_|\\_)\S+` to match both escaped and unescaped underscores after provider prefix - **Added unescape step**: Both patterns now call `.replace(r'\_', '_')` on extracted resource names to normalize output - **Added unit tests**: 10 test cases covering normal, escaped, and mixed underscore scenarios for all three providers ### Example ```python # Before: Failed to match or returned "google\_biglake\_catalog" content = r"# google\_biglake\_catalog" extract_resource_name(content) # Returns None or malformed string # After: Correctly extracts and unescapes extract_resource_name(content) # Returns "google_biglake_catalog" ``` <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > Update to the resource name parsing logic in parser.py. > > Fix to allow for a difference in how the GCP terraform provider markdown files represent services for example google\_biglake\_catalog (with escaped underscores \_) instead of google_biglake_catalog. The parser's regex pattern now handles these escaped underscores. > > ## Implementation Details > > Apply the changes from commit 4ac1b37 which contains the fix for resource name extraction to handle escaped underscores in Azure and GCP patterns. > > This commit should be cherry-picked onto a clean branch based on `dev` to ensure the PR contains only this single focused change. </details> <!-- START COPILOT CODING AGENT SUFFIX --> *This pull request was created from Copilot chat.* > <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: paulJRCurtis <32033064+paulJRCurtis@users.noreply.github.com> Co-authored-by: JBarazani <64306847+JBarazani@users.noreply.github.com>
This contains the policies for service Apigee
Created policies and completed the documentation with markdowns.
moved from form and applied feedback
Created Policies for BigQuery
Resources used:
google_bigquery_dataset
google_bigquery_dataset_access
google_bigquery_dataset_iam
google_bigquery_job
google_bigquery_routine
google_bigquery_row_access_policy
google_bigquery_table
google_bigquery_table_iam
Total Polices: 35
Finalizing policies and inputs for the Apikeys and Biglake service --------- Co-authored-by: Paul Curtis <32033064+paulJRCurtis@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
please review my policies