[D4C Conversion] Converting Compatible D4C Rules to DR

[Aegrah]

Summary

This PR converts all compatible D4C rules to DR. Please review the rules carefully, and also the investigation guides. Any references to specific fields that are unavailable in Defend are removed from the investigation guides. However, this was manual work, thus, please review it carefully.

Conversion constraints

• Due to the reliance on the process.entry_leader.entry_meta.type == "container" field, only process rules were converted, as file and network events do not contain this field.
• Several rules were not converted, as the logic seemed flawed when converting it over to Defend.
• Rules and rule logic were deduplicated or modified to detect the same, with fewer rules or query load.
• The container.security_context.privileged field is unavailable, thus, rules relying on privileged container scenarios were converted without this field set to true. Given the low-volume occurrences of these events, this should not be an issue.

[github-actions]

Rule: Tuning - Guidelines

These guidelines serve as a reminder set of considerations when tuning an existing rule.

Documentation and Context

• Detailed description of the suggested changes.
• Provide example JSON data or screenshots.
• Provide evidence of reducing benign events mistakenly identified as threats (False Positives).
• Provide evidence of enhancing detection of true threats that were previously missed (False Negatives).
• Provide evidence of optimizing resource consumption and execution time of detection rules (Performance).
• Provide evidence of specific environment factors influencing customized rule tuning (Contextual Tuning).
• Provide evidence of improvements made by modifying sensitivity by changing alert triggering thresholds (Threshold Adjustments).
• Provide evidence of refining rules to better detect deviations from typical behavior (Behavioral Tuning).
• Provide evidence of improvements of adjusting rules based on time-based patterns (Temporal Tuning).
• Provide reasoning of adjusting priority or severity levels of alerts (Severity Tuning).
• Provide evidence of improving quality integrity of our data used by detection rules (Data Quality).
• Ensure the tuning includes necessary updates to the release documentation and versioning.

Rule Metadata Checks

• updated_date matches the date of tuning PR merged.
• min_stack_version should support the widest stack versions.
• name and description should be descriptive and not include typos.
• query should be inclusive, not overly exclusive. Review to ensure the original intent of the rule is maintained.

Testing and Validation

• Validate that the tuned rule's performance is satisfactory and does not negatively impact the stack.
• Ensure that the tuned rule has a low false positive rate.

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[w0rk3r]

@Aegrah would push this as BBRs first make sense? Adoption of the endpoint is higher than what we had with D4C, so we can make sure we are not pushing additional noise, while we validate how reliable the fields we didn't use in other rules yet are

[Aegrah]

@w0rk3r I have nothing against moving them to BBR first for a cycle. However, based on the telem available currently, I think none of these rules will be prone to large bulks of FPs. I don't think it's necessary, but at the same time, if we want to minimize the potential risk of pushing a potentially noisy rule, I can make the change! Any thoughts @imays11?

[imays11]

Looks great, thank you Ruben! I understand but don't think it's necessary to move them to BBR first.

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Tests failed:

• ❌ Sensitive Keys Or Passwords Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File Made Executable via Chmod Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ File System Debugger Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Container Management Utility Run Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ AWS Credentials Searched For Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ SSH Process Launched From Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Unusual Interactive Process Launched in a Container (kuery)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Suspicious Network Tool Launched Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Mount Launched Inside a Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Sensitive Files Compression Inside A Container (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[Aegrah]

Thanks for the thorough review @imays11. If this one does not get the approvals necessary to merge prior to my PTO, feel free to merge it before the next release @shashank-elastic!