Skip to content

Conversation

@scunningham
Copy link
Contributor

  • Replace node scope plugin with default. Node scope plugin moved elsewhere.
  • Support experimental field extraction and correlation feature in Prequel product. This feature allows extraction of fields by either regex or jq which can be used to correlate matched terms.

Example rule definition using extractions and a correlation:

rules:
  - cre:
      id: TestSuccessSimpleExtraction
    metadata:
      id: "J7uRQTGpGMyL1iFpssnBeS"
      hash: "rdJLgqYgkEp8jg8Qks1qiq"
      generation: 1
    rule:
      sequence:
        window: 30s
        event:
          source: log
        correlations:
          - corr1
        order:
          - value: "term1"
            extract:
            - name: "corr1"
              jq: ".field1"
          - value: "term2"
            extract:
              - name: "corr1"
                jq: ".field1"

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for field extraction and correlation features in the Prequel compiler. The changes enable extracting fields using either regex or jq expressions, which can then be correlated across matched terms in sequence rules.

Key changes:

  • Renamed NodePlugin to DefaultPlugin and updated scope from "node" to "default"
  • Added extraction support with validation for extract names (alphanumeric and underscores)
  • Enhanced negate field validation to prevent count > 1 and disallow extracts

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
pkg/testdata/rules.go Added test case TestSuccessSimpleExtraction demonstrating extraction and correlation syntax
pkg/schema/schema.go Added ScopeDefault constant to replace node scope
pkg/parser/tree.go Added ExtractT struct, extraction validation logic, and error types for extract name validation
pkg/parser/parse.go Added ParseExtractT struct to support parsing extract fields from YAML
pkg/compiler/plugin.go Renamed NodePlugin to DefaultPlugin with corresponding constructor
pkg/compiler/compiler.go Updated default plugin instantiation and scope mapping to use ScopeDefault
pkg/ast/ast_test.go Added test case for simple extraction feature
pkg/ast/ast_log.go Implemented extraction term processing, correlation support, and negate field validation
pkg/ast/ast.go Added AstExtractT struct and updated AstFieldT to include extracts

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Replace node scope plugin with default.  Node scope plugin moved to machine.
@scunningham scunningham merged commit 6d9c613 into main Oct 17, 2025
1 check passed
@scunningham scunningham deleted the correlations branch October 17, 2025 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants