Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filebeat/input/syslog: Improve Cisco log parsing #37748

Conversation

taylor-swanson
Copy link
Contributor

@taylor-swanson taylor-swanson commented Jan 25, 2024

Proposed commit message

  • Improved cisco log parsing in the syslog input to handle both sequence number and hostname
  • Added tests

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Unit tests:

  • Run the unit tests in filebeat/input/syslog

System test:

  • Enable the Cisco module and enable the syslog input on either UDP or TCP.
$ filebeat modules enable cisco

# Add to modules.d/cisco.yml
- module: cisco
  ios:
    enabled: true
    var.input: syslog
    var.syslog_host: localhost
    var.syslog_port: 9000
    var.syslog_protocol: udp
  • Send sample logs to the listening port and verify expected data.
<190>589265: TESTHOST1234: Feb 8 18:55:31.306: %SEC-11-IPACCESSLOGP: list 177 denied udp 10.0.0.1(53640) -> 10.100.0.1(15600), 1 packet

Related issues

  • Relates elastic/sdh-beats#4295

@taylor-swanson taylor-swanson self-assigned this Jan 25, 2024
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 25, 2024
@taylor-swanson taylor-swanson added bug and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jan 25, 2024
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 25, 2024
@taylor-swanson taylor-swanson added backport-v8.12.0 Automated backport with mergify Team:Security-Deployment and Devices Deployment and Devices Team in Security Solution Team:Security-External Integrations and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jan 25, 2024
Copy link
Contributor

mergify bot commented Jan 25, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b bugfix/syslog-input-parsing upstream/bugfix/syslog-input-parsing
git merge upstream/main
git push upstream bugfix/syslog-input-parsing

@taylor-swanson taylor-swanson force-pushed the bugfix/syslog-input-parsing branch from 85aac18 to 3b83e7e Compare January 25, 2024 15:27
@elasticmachine
Copy link
Collaborator

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 7 min 42 sec

Pipeline error 1

This error is likely related to the pipeline itself. Click here
and then you will see the error (either incorrect syntax or an invalid configuration).

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@taylor-swanson taylor-swanson changed the title Improve Cisco log parsing in syslog input filebeat/input/syslog: Improve Cisco log parsing Jan 25, 2024
@elasticmachine
Copy link
Collaborator

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 118 min 31 sec

Pipeline error 1

This error is likely related to the pipeline itself. Click here
and then you will see the error (either incorrect syntax or an invalid configuration).

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

- Improved cisco log parsing to handle both sequence number and hostname
- Added tests
@taylor-swanson taylor-swanson force-pushed the bugfix/syslog-input-parsing branch from 462be06 to aed82f3 Compare January 26, 2024 16:26
@elasticmachine
Copy link
Collaborator

❕ Build Aborted

There is a new build on-going so the previous on-going builds have been aborted.

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Start Time: 2024-01-26T16:11:54.428+0000

  • Duration: 30 min 7 sec

Test stats 🧪

Test Results
Failed 3
Passed 2311
Skipped 203
Total 2517

Test errors 3

Expand to view the tests failures

Build&Test / x-pack/filebeat-goIntegTest / [empty] – TEST-go-integration.xml
  • no error details
  • Expand to view the stacktrace

     Test report file /var/lib/jenkins/workspace/PR-37748-4-8bee37f8-a1bd-4757-9353-3496dd1079e1/src/github.com/elastic/beats/build/x-pack/filebeat/build/TEST-go-integration.xml was length 0 
    

Build&Test / filebeat-goIntegTest / [empty] – TEST-go-integration.xml
  • no error details
  • Expand to view the stacktrace

     Test report file /var/lib/jenkins/workspace/PR-37748-4-48cb05ee-343c-4ef1-9aa1-ef6ae63bb3d9/src/github.com/elastic/beats/build/filebeat/build/TEST-go-integration.xml was length 0 
    

Build&Test / x-pack/filebeat-unitTest / [empty] – TEST-go-unit.xml
  • no error details
  • Expand to view the stacktrace

     Test report file /var/lib/jenkins/workspace/PR-37748-4-6c5f24cb-4630-4437-8dc4-d73122344acc/src/github.com/elastic/beats/build/x-pack/filebeat/build/TEST-go-unit.xml was length 0 
    

Steps errors 1

Expand to view the steps failures

Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'org.jenkinsci.plugins.workflow.steps.FlowInterruptedException'

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 131 min 49 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@taylor-swanson
Copy link
Contributor Author

After some internal discussions, I've decided to abandon this change. It was discovered that there are several more formats that aren't covered by the change, and we would need to support all of these cases. Supporting these formats would require more extensive changes to the parser. Considering that the input is going to be deprecated soon, there are better alternatives to explore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v8.12.0 Automated backport with mergify bug enhancement Filebeat Filebeat Team:Security-Deployment and Devices Deployment and Devices Team in Security Solution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants