[DaC] Beta Release #3889
Merged
[DaC] Beta Release #3889
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* [FR] Add custom rule directory support * [FR] Add support for configurable tests and validation * [FR] Add support to decouple actions and exceptions * update actions schema * add custom-rules init-config command * update docs --------- Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com>
…3654) * Improve dac custom init * Fix path naming * patch for ci runs * add doc strings and rename test config name * expand how unit test are selected * Updated to support list of dirs * raise unlink to CLI * Fix unit test config post assertion * Add a custom method to generate the test config * add explicit checks for package.yml fields * newline * raise SystemExit instead of sys.exit * Collapsing missing config message and exit * flake8 * update base config * typo * Updated config parsing * Update detection_rules/config.py Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com> * simplify package requirements * remove import * add dataclass to validate rules config file and create default setup-config cli * add kibana_version cli param * update doc string * rename delete cli option to overwrite, and small edits to exceptions * Typo in config * Add resolve * Added TODO * Cleanup * Update path to config path * Added get_rules_dir_path function * revert config change * Updated config * Minor Cleanup * Cleanup get_base_rule_dir * Updated to remove try except * update test cli command * Updated config generation * Add default in config * Update test CLI * update readme * readme updates * Add support for multiple rules dirs * Updated import-rules-to-repo readme * Update unit test * Remove redundant check * Remove additional parse_rules_config from mappings --------- Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co> Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com>
optional BBR rules
* Add flag to enable all prebuilt tests * Add additional bypasses * Added format_test_string function
|
There is an intermittent issue when importing many rules and exception lists at the same time related to elastic/kibana#143864 that is currently being investigated. This may cause exception list references to be stripped when importing rules in some cases. |
| # 2.0; you may not use this file except in compliance with the Elastic License | ||
| # 2.0. | ||
|
|
||
| """Dataclasses for Action.""" |
Mikaayenson
approved these changes
Aug 6, 2024
There was a problem hiding this comment.
🚢
A lot of effort in this PR to support many edge cases. Based on the walkthroughs, test runs, re: import/export via ndjson / kibana on multiple rules (sigma, internal team rules, and past exercise rulesets), this lgtm.
Most of the nuances have been documented in dac-reference so I think this will help most of the users get up and running.
We know that there are a couple gaps like schema support for new kibana features (alert suppression on specific rule types, etc.). We will tackle that in a different/later PR.
Final tests on the version lock didn't have any changes, so I think we're good with our release branches, but we'll see when we merge.
Merging this formally moves us to beta where we'll continue edge case testing with reduced hurdles for users to onboard and start natively integrating into their workflows.
We'll also reduce feature development for a bit to ensure this existing capability works as expected given our diverse user base and environments.
🚀
protectionsmachine
pushed a commit
that referenced
this pull request
Aug 6, 2024
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com> Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com> Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co> (cherry picked from commit 47d7a3a)
protectionsmachine
pushed a commit
that referenced
this pull request
Aug 6, 2024
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com> Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com> Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co> (cherry picked from commit 47d7a3a)
protectionsmachine
pushed a commit
that referenced
this pull request
Aug 6, 2024
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com> Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com> Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co> (cherry picked from commit 47d7a3a)
protectionsmachine
pushed a commit
that referenced
this pull request
Aug 6, 2024
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com> Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com> Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co> (cherry picked from commit 47d7a3a)
protectionsmachine
pushed a commit
that referenced
this pull request
Aug 6, 2024
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com> Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com> Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co> (cherry picked from commit 47d7a3a)
protectionsmachine
pushed a commit
that referenced
this pull request
Aug 6, 2024
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com> Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com> Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co> (cherry picked from commit 47d7a3a)
This was referenced Aug 7, 2024
Closed
zsohamwag
pushed a commit
to zsohamwag/zsoham-detection-rules
that referenced
this pull request
Sep 13, 2024
Co-authored-by: Justin Ibarra <16747370+brokensound77@users.noreply.github.com> Co-authored-by: brokensound77 <brokensound77@users.noreply.github.com> Co-authored-by: Mika Ayenson <Mikaayenson@users.noreply.github.com> Co-authored-by: Mika Ayenson <mika.ayenson@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request
Issue link(s):
https://github.com/elastic/ia-trade-team/issues/345
Summary - What I changed
Note: There is an intermittent issue when importing many rules and exception lists at the same time related to elastic/kibana#143864 that is currently being investigated. This may cause exception list references to be stripped when importing rules in some cases.
This PR is to promote the DAC-Feature branch to a beta release through merging to main.
We will want this PR to go through extensive, independent testing. One of the goals of pushing to Beta is to feature freeze the current DaC features. Testing the completeness of the features to achieve the desired workflow of using DaC to manage rules is just as important as functionality testing.
Note: Make sure to especially review logic changes in core functions like
multi_collection(f):TLDR: Jump to automated testing section and run tests, with these tests and manual review should be good to merge.
Testing
For testing we have 2 main workflows we need to test.
kibana import/export-rulesimport-rules-to-repo and export-rules-to-repoThese workflows should support appropriate testing and configuration for managing unit tests and schema validation.
Examples of things that we need to test for each workflow.
We want to know where our code breaks to make sure it breaks gracefully on cases where that is appropriate. We should try to load and rules that are known to be broken, missing data, missing indexes, etc. that expect to fail as we want to make sure we are controlling the way they fail.
With this in mind there are a few collections of rules that we can use for testing.
Major Functionality Updates
Configuration Files
We need to test what happens when we have bad settings in the config files or when they are missing
Commands with major functionality updates that should be tested
Specifically these have been updated with support for the configuration files and support for handling exceptions and need to be tested.
kibana export-ruleskibana import-rulesimport-rules-to-repoexport-rules-from-repoAutomated Testing
Base Functionality Testing (to test core features did not break)
make test-climake test-remote-cliNew Feature Functionality Testing (DAC-feature additions)
test_custom_cli.bash
test_custom_remote_cli.bash
python -m detection_rules custom-rules setup-config custom_rules_testingbash test_custom_cli.sh <path_to_ndjson_file>bash test_custom_remote_cli.shAutomated Testing Expected Results
Please note that your testing results will be different depending on your config and the rules you used to test. To obtain the desired results you will need to match the configuration for each command. The results for
make test-clirequire a non-custom config as these commands are designed to test our base functionality for detection rules.make test-remote-clican function either with a custom config or without one; however, the results will vary based on your custom rules. It may be common for custom rules to have a version and a revision field set which will cause them all to fail unless bypass_version_lock set toTrue. The same is true for both custom bash scripts that are designed to test DaC feature, and expect a custom config to be set.make test-cli
make test-remote-cli
test_custom_cli.bash
test_custom_remote_cli.bash
Checklist
bug,enhancement,Rule: New,Rule: Deprecation,Rule: Promote,Rule: Tuning,Hunt: New, orHunt: Tuningso guidelines can be generatedmeta:rapid-mergelabel if planning to merge within 24 hours