Skip to content

CONNECTORS.md

Latest commit

 

History

History
197 lines (152 loc) · 5.07 KB

CONNECTORS.md

File metadata and controls

197 lines (152 loc) · 5.07 KB

Creating Custom Connectors

Overview

Connectors import telemetry and evidence from security tools (SIEM, EDR, logs) to validate detections.

Connector Interface

Every connector should implement:

(ns purpleops-bas.connectors.my-connector
  (:require
   [clj-http.client :as http]
   [cheshire.core :as json]
   [taoensso.timbre :as log]))

;; Parse raw event into standard format
(defn parse-event [raw-event]
  {:timestamp ...
   :source "MyTool"
   :event-type ...
   :raw-data raw-event})

;; Import from file
(defn import-from-file [filepath]
  (map parse-event (load-data filepath)))

;; Import from API
(defn import-from-api [config]
  (map parse-event (query-api config)))

Standard Event Format

All connectors should return events in this format:

{:timestamp #inst "2026-01-06T10:30:00Z"
 :source "SourceName"          ;; SIEM, EDR, Firewall, etc.
 :event-type "EventType"        ;; Alert, Detection, Log, etc.
 :severity "high"               ;; Optional
 :confidence 0.85               ;; Optional
 :fields {...}                  ;; Tool-specific fields
 :raw-data {...}}               ;; Original event

Example: Custom SIEM Connector

(ns purpleops-bas.connectors.custom-siem
  (:require
   [clj-http.client :as http]
   [cheshire.core :as json]
   [taoensso.timbre :as log]))

(defn parse-custom-event [event]
  {:timestamp (:event_time event)
   :source "CustomSIEM"
   :event-type (:alert_type event)
   :severity (:severity event)
   :fields {:rule-name (:rule event)
            :src-ip (:src_ip event)
            :dst-ip (:dst_ip event)}
   :raw-data event})

(defn import-from-api [{:keys [base-url api-key query]}]
  (try
    (let [response (http/post (str base-url "/api/v1/search")
                              {:headers {"X-API-Key" api-key}
                               :body (json/generate-string {:query query})
                               :content-type :json
                               :as :json})]
      (map parse-custom-event (get-in response [:body :results])))
    (catch Exception e
      (log/error e "Failed to query Custom SIEM")
      [])))

(defn import-from-file [filepath]
  (try
    (let [data (-> filepath slurp (json/parse-string true))]
      (map parse-custom-event (:events data)))
    (catch Exception e
      (log/error e "Failed to import from file")
      [])))

Integrating Your Connector

  1. Add namespace to connectors/src/purpleops_bas/connectors/

  2. Register in orchestrator - modify orchestrator/src/.../critic.clj:

(require '[purpleops-bas.connectors.custom-siem :as custom-siem])

(defn import-detections [run-id sources]
  (concat
   (when (contains? sources :custom-siem)
     (custom-siem/import-from-api {...}))
   ;; other sources
   ))
  1. Test import:
(custom-siem/import-from-file "test-data/events.json")

Built-in Connectors

  • SIEM: purpleops-bas.connectors.siem - Splunk, Elastic
  • EDR: purpleops-bas.connectors.edr - Defender, CrowdStrike
  • Logs: purpleops-bas.connectors.logs - syslog, Windows Event, CloudTrail

API Authentication

Store credentials securely:

;; Use environment variables
(def api-key (System/getenv "CUSTOM_SIEM_API_KEY"))

;; Or load from config
(def config (load-config))
(def api-key (get-in config [:connectors :custom-siem :api-key]))

Rate Limiting

Implement rate limiting for API calls:

(def rate-limiter (atom {:last-call 0 :min-interval-ms 1000}))

(defn rate-limited-call [f]
  (let [now (System/currentTimeMillis)
        last-call (:last-call @rate-limiter)
        interval (:min-interval-ms @rate-limiter)]
    (when (< (- now last-call) interval)
      (Thread/sleep (- interval (- now last-call))))
    (swap! rate-limiter assoc :last-call (System/currentTimeMillis))
    (f)))

Error Handling

Always handle errors gracefully:

(defn import-with-retry [import-fn retries]
  (loop [attempts retries]
    (try
      (import-fn)
      (catch Exception e
        (if (> attempts 1)
          (do
            (log/warn "Import failed, retrying..." e)
            (Thread/sleep 2000)
            (recur (dec attempts)))
          (do
            (log/error e "Import failed after retries")
            []))))))

Testing Your Connector

# In REPL
clj -M:dev

(require '[purpleops-bas.connectors.custom-siem :as cs])

;; Test file import
(cs/import-from-file "test-data/sample.json")

;; Test API import (with test credentials)
(cs/import-from-api {:base-url "https://test.siem.local"
                     :api-key "test-key"
                     :query "search index=main"})

Best Practices

  1. Parse consistently - use standard event format
  2. Handle missing fields - provide defaults
  3. Log errors - use timbre for logging
  4. Test with real data - validate parsing logic
  5. Document fields - explain what each field means
  6. Version compatibility - handle API changes

Need Help?