Merge branch 'main' into jenworthington-patch-5

.github/ISSUE_TEMPLATE/bug-escalation-report.md

@@ -11,7 +11,11 @@ assignees: ''
 
 **What is the sc4s version ?**
 
-**Is there a pcap available?**
+**Which operating system (including its version) are you using for hosting SC4S?**
+
+**Which runtime (Docker, Podman, Docker Swarm, BYOE, MicroK8s) are you using for SC4S?**
+
+**Is there a pcap available? If so, would you prefer to attach it to this issue or send it to Splunk support?**
 
 **Is the issue related to the environment of the customer or Software related issue?**
 

.github/ISSUE_TEMPLATE/feature-request-e-g--a-new-filter--a-new-processor-etc--.md

@@ -7,16 +7,18 @@ assignees: ''
 
 ---
 
-**What is the sc4s version ?**
+**What is the sc4s version?**
 
-**Is there a pcap available?**
+**Is there a pcap available? If so, would you prefer to attach it to this issue or send it to Splunk support?**
 
 **What the vendor name?**
 
 **What's the product name?**
 
-** Feature Request description: **
+**If you're requesting support for a new vendor, do you have any preferences regarding the default index and sourcetype for their events?**
 
-** Should it support TCP or UDP?**
+**Do you have syslog documentation or a manual for that device??**
 
-** Do you want to have it for local usage or prepare a github PR? **
+**Feature Request description:**
+
+**Do you want to have it for local usage or prepare a github PR?**

.github/workflows/agreements.yaml

@@ -18,7 +18,7 @@ jobs:
       - name: "CLA Assistant"
         if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA') || github.event_name == 'pull_request_target'
         # Alpha Release
-        uses: contributor-assistant/github-action@v2.3.1
+        uses: contributor-assistant/github-action@v2.4.0
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           # the below token should have repo scope and must be manually added by you in the repository's secret
@@ -43,7 +43,7 @@ jobs:
       - name: "COC Assistant"
         if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read the Code of Conduct and I hereby accept the Terms') || github.event_name == 'pull_request_target'
         # Alpha Release
-        uses: contributor-assistant/github-action@v2.3.1
+        uses: contributor-assistant/github-action@v2.4.0
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           # the below token should have repo scope and must be manually added by you in the repository's secret

.github/workflows/cd-baremtal.yaml

@@ -48,6 +48,6 @@ jobs:
           tar rvf /tmp/baremetal.tar -C package/sbin entrypoint.sh
           sha512sum -b /tmp/baremetal.tar > /tmp/baremetal_checksum.txt
       - name: Release
-        uses: softprops/action-gh-release@v1
+        uses: softprops/action-gh-release@v2
         with:
           files: /tmp/baremetal.tar, /tmp/baremetal_checksum.txt

.github/workflows/cd-docs-pdf.yaml

@@ -42,6 +42,6 @@ jobs:
           sudo apt-get install -y pandoc texlive librsvg2-bin texlive-latex-extra
           pandoc -s --pdf-engine=pdflatex -o /tmp/SC4S_docs.pdf $(find . -type f -name "*.md")
       - name: Release
-        uses: softprops/action-gh-release@v1
+        uses: softprops/action-gh-release@v2
         with:
           files: /tmp/SC4S_docs.pdf

.github/workflows/cd-oci-container.yaml

@@ -43,6 +43,6 @@ jobs:
           sha512sum -b /tmp/oci_container.tar.gz > /tmp/oci_container_checksum.txt
 
       - name: Release
-        uses: softprops/action-gh-release@v1
+        uses: softprops/action-gh-release@v2
         with:
           files: /tmp/oci_container.tar.gz, /tmp/oci_container_checksum.txt

.github/workflows/cd-pages.yaml

@@ -23,7 +23,7 @@ jobs:
         uses: actions/checkout@v4
         with:
           fetch-depth: 0
-      - uses: oleksiyrudenko/gha-git-credentials@v2.1.1
+      - uses: oleksiyrudenko/gha-git-credentials@v2.1.2
         with:
           token: "${{ secrets.GITHUB_TOKEN }}"
       - uses: actions/setup-python@v5
@@ -46,7 +46,7 @@ jobs:
         with:
           fetch-depth: 0
       - name: Install Helm
-        uses: azure/setup-helm@v3
+        uses: azure/setup-helm@v4
         with:
           version: v3.11.2
       - name: Upload
@@ -63,7 +63,7 @@ jobs:
           helm repo index /tmp/package --url https://github.com/splunk/splunk-connect-for-syslog/releases/download/$VERSION --merge /tmp/origin/index.yaml
           cp /tmp/package/index.yaml /tmp/index/
       - name: Deploy
-        uses: peaceiris/actions-gh-pages@v3
+        uses: peaceiris/actions-gh-pages@v4
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
           publish_dir: /tmp/index

.github/workflows/ci-lite.yaml

@@ -253,7 +253,7 @@ jobs:
       image: python:3.9-buster
     services:
       splunk:
-        image: splunk/splunk:9.2.0
+        image: splunk/splunk:${{ fromJson(needs.meta.outputs.matrix_supportedSplunk)[0].version }}
         ports:
           - 8088:8088
           - 8089:8089
@@ -305,7 +305,7 @@ jobs:
       image: python:3.9-buster
     services:
       splunk:
-        image: splunk/splunk:9.2.0
+        image: splunk/splunk:${{ fromJson(needs.meta.outputs.matrix_supportedSplunk)[0].version }}
         ports:
           - 8088:8088
           - 8089:8089

.github/workflows/ci-main.yaml

@@ -255,7 +255,7 @@ jobs:
       image: python:3.9-buster
     services:
       splunk:
-        image: splunk/splunk:9.2.0
+        image: splunk/splunk:${{ fromJson(needs.meta.outputs.matrix_supportedSplunk)[0].version }}
         ports:
           - 8088:8088
           - 8089:8089
@@ -307,7 +307,7 @@ jobs:
       image: python:3.9-buster
     services:
       splunk:
-        image: splunk/splunk:9.2.0
+        image: splunk/splunk:${{ fromJson(needs.meta.outputs.matrix_supportedSplunk)[0].version }}
         ports:
           - 8088:8088
           - 8089:8089
@@ -357,7 +357,7 @@ jobs:
         uses: actions/checkout@v4
         with:
           fetch-depth: 0
-      - uses: oleksiyrudenko/gha-git-credentials@v2.1.1
+      - uses: oleksiyrudenko/gha-git-credentials@v2.1.2
         with:
           token: "${{ secrets.GITHUB_TOKEN }}"
       - uses: actions/setup-python@v5

ansible/playbooks/microk8s.yml

@@ -12,5 +12,6 @@
       openebs: true
   tasks:
     - include_tasks: ../tasks/mk8s/install_mk8s.yml
+    - include_tasks: ../tasks/mk8s/deploy_secrets.yml
     - include_tasks: ../tasks/mk8s/install_helm_repo.yml
     - include_tasks: ../tasks/mk8s/deploy_app.yml

ansible/playbooks/microk8s_ha.yml

@@ -12,6 +12,7 @@
       openebs: true
   tasks:
     - include_tasks: ../tasks/mk8s/install_mk8s.yml
+    - include_tasks: ../tasks/mk8s/deploy_secrets.yml
     - include_tasks: ../tasks/mk8s/install_helm_repo.yml
     - include_tasks: ../tasks/mk8s/deploy_app.yml
     - include_tasks: ../tasks/mk8s/get_registration_token.yml

ansible/tasks/mk8s/deploy_secrets.yml

@@ -0,0 +1,25 @@
+---
+- name: Load k8s secrets
+  include_vars:
+    file: "{{ item }}"
+  with_first_found:
+    - files:
+      - /opt/ansible/resources/k8s_secrets.yaml
+      - /opt/charts/splunk-connect-for-syslog/secrets.yaml
+
+- name: Create mTLS secret
+  ansible.builtin.shell: |
+    microk8s kubectl apply -f - <<EOF
+    apiVersion: v1
+    kind: Secret
+    metadata:
+      name: {{ hec_tls.secret }}
+    type: Opaque
+    data:
+      key.pem: {{ hec_tls.value.key | b64encode }}
+      cert.pem: {{ hec_tls.value.cert | b64encode }}
+      ca_cert.pem: {{ hec_tls.value.ca | b64encode }}
+    EOF
+  when:
+    - hec_tls is defined
+    - ('secret' in hec_tls) and ('value' in hec_tls)

charts/splunk-connect-for-syslog/Chart.yaml

@@ -2,5 +2,5 @@ apiVersion: v2
 name: splunk-connect-for-syslog
 description: Deploy Splunk Connect for Syslog
 type: application
-version: 3.23.2
-appVersion: "3.23.2"
+version: 3.25.0
+appVersion: "3.25.0"

charts/splunk-connect-for-syslog/templates/statefulset.yaml

@@ -65,6 +65,10 @@ spec:
           - name: SC4S_DEST_SPLUNK_HEC_DEFAULT_DISKBUFF_ENABLE
             value: "no"
           {{- end }}
+          {{- if .Values.splunk.hec_tls }}
+          - name: SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_MOUNT
+            value: "/etc/syslog-ng/tls/hec"
+          {{- end }}
           {{- if .Values.sc4s.existingCert }}
           - name: SC4S_SOURCE_TLS_ENABLE
             value: "yes"
@@ -186,6 +190,11 @@ spec:
           - name: data
             mountPath: /var/lib/syslog-ng/
           {{- end }}
+          {{- if .Values.splunk.hec_tls }}
+          - name: hec-tls
+            mountPath: /etc/syslog-ng/tls/hec/
+            readOnly: true
+          {{- end }}
           {{- if .Values.sc4s.existingCert }}
           - name: tls
             mountPath: /etc/syslog-ng/tls/
@@ -224,6 +233,11 @@ spec:
           resources:
             {{- toYaml .Values.resources | nindent 12 }}
       volumes:
+        {{- if .Values.splunk.hec_tls }}
+        - name: hec-tls
+          secret:
+            secretName: {{ .Values.splunk.hec_tls }}
+        {{- end }}
         {{- if .Values.sc4s.existingCert }}
         - name: tls
           secret:

docs/dashboard.md

@@ -5,19 +5,22 @@ The SC4S Metrics and Events Dashboard lets you monitor crucial metrics and event
 ## Functionalities
 
 ### Overview metrics
+![Overview metrics](dashboard_overview_metrics.png)
 The dashboard displays the cumulative sum of received and dropped messages for all SC4S instances in a chosen interval and for the specified time range. By default the interval is set to 30 seconds and the time range is 15 minutes.
 
 The Received Messages panel can be used as a heartbeat metric. A healthy SC4S instance should send at least one message per 30 seconds. This metrics message is included in the count.
 
 The Dropped Messages panel should remain at a constant level of 0. If SC4S drops messages due to filters, slow performance, or for any other reason, the number of dropped messages will persist until the instance restarts. This panel does not include potential UDP messages dropped from the port buffer, which SC4S is not able to track.
 
 ### Single instance metrics
+![Single instance metrics](dashboard_single_instance_metrics.png)
 You can display the instance name and SC4S version for a chosen SC4S instance.
 SC4S is available in versions greater than or equal to 3.16.0.
 
 The dashboard also displays a timechart of deltas for received, queued, and dropped messages for a chosen SC4S instance.
 
 ### Single instance events
+![Single instance events](dashboard_single_instance_events.png)
 The dashboard helps to analyze traffic processed by an SC4S instance by visualizing the following events data:
 
 - total number of events

docs/edge_processor.md

@@ -0,0 +1,113 @@
+# Edge Processor integration guide (Experimental)
+
+## Intro
+
+You can use the `Edge Processor` to:
+
+* Enrich log messages with extra data, such as adding a new field or overriding an index using `SPL2`.
+* Filter log messages using `SPL2`.
+* Send log messages to alternative destinations, for example, `AWS S3` or `Apache Kafka`.
+
+## How it works
+
+```mermaid
+stateDiagram
+    direction LR
+
+    SC4S: SC4S
+    EP: Edge Processor
+    Dest: Another destination
+    Device: Your device
+    S3: AWS S3
+    Instance: Instance
+    Pipeline: Pipeline with SPL2
+
+    Device --> SC4S: Syslog protocol
+    SC4S --> EP: HEC
+    state EP {
+      direction LR
+      Instance --> Pipeline
+    }
+    EP --> Splunk
+    EP --> S3
+    EP --> Dest
+```
+
+## Set up the Edge Processor for SC4S
+
+### Set up on Docker / Podman
+
+1. On the `env_file`, configure the HEC URL as IP of *managed instance*, that you registered on Edge Processor.
+2.  Add your HEC token. You can find your token in the Edge Processor "global settings" page. 
+
+```
+SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=http://x.x.x.x:8088
+SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+```
+
+### Kubernetes
+
+1. Set up the Edge Processor on your `values.yaml` HEC URL using the IP of *managed instance*, that you registered on Edge Processor.
+
+2. Provide the hec_token. You can find this token on the Edge Processor's "global settings" page.
+
+```
+splunk:
+  hec_url: "http://x.x.x.x:8088"
+  hec_token: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
+```
+
+## mTLS encryption
+
+### Prepare your certificates
+
+Before setup, [generate mTLS certificates](https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/EdgeProcessor/SecureForwarders). Server mTLS certificates should be uploaded to `Edge Processor` and client certifcates should be used with `SC4S`.
+
+Rename the certificate files. SC4S requires the following names:
+
+  * `key.pem` - client certificate key
+  * `cert.pem` - client certificate
+  * `ca_cert.pem` - certificate authority
+
+### Docker / Podman
+
+  1. Use HTTPS in HEC url: `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://x.x.x.x:8088`.
+  2. Move your clients mTLS certificates to `/opt/sc4s/tls/hec`.
+  3. Mount `/opt/sc4s/tls/hec` to `/etc/syslog-ng/tls/hec` using docker/podman volumes.
+  4. Define mounting mTLS point for HEC: `SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_MOUNT=/etc/syslog-ng/tls/hec`.
+  5. Start or restart SC4S.
+
+### Kubernetes
+
+  1. Add the secret name of the mTLS certificates to the `values.yaml` file:
+
+```
+splunk:
+  hec_url: "https://x.x.x.x:8088"
+  hec_token: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
+  hec_tls: "hec-tls-secret"
+```
+
+  2. Add your mTLS certificates to the `charts/splunk-connect-for-syslog/secrets.yaml` file:
+
+```
+hec_tls:
+  secret: "hec-tls-secret"
+  value:
+    key: |
+      -----BEGIN PRIVATE KEY-----
+      Exmaple key
+      -----END PRIVATE KEY-----
+    cert: |
+      -----BEGIN CERTIFICATE-----
+      Exmaple cert
+      -----END CERTIFICATE-----
+    ca: |
+      -----BEGIN CERTIFICATE-----
+      Example ca
+      -----END CERTIFICATE-----
+```
+
+  3. Encrypt your `secrets.yaml` using `ansible-vault encrypt charts/splunk-connect-for-syslog/secrets.yaml`.
+  4. Add the IP address for your cluster nodes to the inventory file `ansible/inventory/inventory_microk8s_ha.yaml`.
+  5. Deploy the Ansible playbook `ansible-playbook -i ansible/inventory/inventory_microk8s_ha.yaml ansible/playbooks/microk8s_ha.yml --ask-vault-pass`