Wirego v2 (ZMQ) (#19)

* Setting up WIrego ZMQ plugin * Trying with ZMQ (not czmq) * Wireshark plugin is now "wirego_bridge" * Added version check A bit of cleanup * Much implementation. Not mucho tests. * More stable * More cb implmentations WIP: dissect * More implem * Backported dissect * Info and protocol * Backport completed. Needs a lot a testing and refactoring. * Cleanup & refactoring * Wireshark now loads properly. Still many things to fix * Several FIX and comments in go package * Added missing source file * Using slog instead of log in order to setup verbosity level * Dissect fixed. basic use case seems to be working. * Setup timeouts * Be less verbose by default * Don't need to duplicate Wireshark's payload buffers anymore Use ipc:// by default * Updated examples Updated examples's README Added MIGRATIOn documentation for migrating from v1 to v2. * Updated documentation * Added PROTOCOL documentation. * Refer to the ZMQ protocol spec from the main README * Reorganizing project structure * Make the logo a bit smaller. * Cosmetics * Functional tests for Go package * More tests * GetProtocol and info * All functional tests for Go package. * A bit of function and ZMQ calls renaming for better clarity. * Renaming in go package * FIX in heur_detection to match new packet.h header * Added changelog * CI updates * CI cleanup * CI FIXES: - link on macos with homebrew - support Wireshark heur_dissector_add proto update between 4.2 and 4.4 * DIsable Macos build for WS 4.0 --------- Co-authored-by: Neb <bgirard@quarkslab.com>
quarkslab · Dec 24, 2024 · 829487d · 829487d
1 parent b964e68
commit 829487d
Show file tree

Hide file tree

Showing 60 changed files with 3,318 additions and 2,105 deletions.
diff --git a/.github/workflows/cmake-macos.yml b/.github/workflows/cmake-macos.yml
@@ -12,15 +12,15 @@ jobs:
     runs-on: macos-latest
     strategy:
       matrix:
-        wireshark_version: ["4.0", "4.2"]
+        wireshark_version: ["4.2", "4.4"]
     steps:
       - name: Set output folder
         env:
           VERSION: ${{ matrix.wireshark_version }}
         run: echo "OUTPUT_PATH=${VERSION//./-}" >> $GITHUB_ENV
 
       - name: Install deps
-        run: brew install c-ares glib libgcrypt pcre2 speexdsp
+        run: brew install c-ares glib libgcrypt pcre2 speexdsp zeromq
 
       - name: Clone Wirego plugin
         uses: actions/checkout@v3
@@ -35,7 +35,7 @@ jobs:
           ref: release-${{ matrix.wireshark_version }}
 
       - name: Create plugin link
-        run: ln -s ${{ github.workspace }}/wirego/wirego_plugin ${{ github.workspace }}/wireshark/plugins/epan/wirego
+        run: ln -s ${{ github.workspace }}/wirego/wirego_bridge ${{ github.workspace }}/wireshark/plugins/epan/wirego
 
       - name: Create output dir
         run: mkdir ${{ github.workspace }}/build

diff --git a/.github/workflows/cmake-ubuntu.yml b/.github/workflows/cmake-ubuntu.yml
@@ -12,10 +12,10 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        wireshark_version: ["4.0", "4.2"]
+        wireshark_version: ["4.0", "4.2", "4.4"]
     steps:
       - name: Install wireshark deps on linux
-        run: sudo apt-get install -y git build-essential cmake libgcrypt-dev libpcap-dev libgcrypt20-dev libglib2.0-dev flex bison libpcre2-dev libnghttp2-dev libc-ares-dev libspeexdsp-dev
+        run: sudo apt-get install -y git build-essential cmake libgcrypt-dev libpcap-dev libgcrypt20-dev libglib2.0-dev flex bison libpcre2-dev libnghttp2-dev libc-ares-dev libspeexdsp-dev libzmq5-dev
 
       - name: Clone Wirego plugin
         uses: actions/checkout@v3
@@ -30,7 +30,7 @@ jobs:
           ref: release-${{ matrix.wireshark_version }}
 
       - name: Create plugin link
-        run: ln -s ${{ github.workspace }}/wirego/wirego_plugin ${{ github.workspace }}/wireshark/plugins/epan/wirego
+        run: ln -s ${{ github.workspace }}/wirego/wirego_bridge ${{ github.workspace }}/wireshark/plugins/epan/wirego
 
       - name: Create output dir
         run: mkdir ${{ github.workspace }}/build

diff --git a/.github/workflows/cmake-windows.yml b/.github/workflows/cmake-windows.yml
@@ -12,13 +12,13 @@ jobs:
     runs-on: windows-2022
     strategy:
       matrix:
-        wireshark_version: ["4.0", "4.2"]
+        wireshark_version: ["4.0", "4.2", "4.4"]
     env:
       PLATFORM: x64
       WIRESHARK_BASE_DIR: C:\Development
     steps:
       - name: Choco install
-        run: choco install -y --force --no-progress asciidoctorj xsltproc docbook-bundle nsis winflexbison3 cmake
+        run: choco install -y --force --no-progress asciidoctorj xsltproc docbook-bundle nsis winflexbison3 cmake zeromq
       - name: Add msbuild to PATH
         uses: microsoft/setup-msbuild@v1.1
         with:
@@ -40,7 +40,7 @@ jobs:
       - name: Create plugin dir
         run: mkdir ${{ github.workspace }}\wireshark\plugins\epan\wirego
       - name: Copy plugin files
-        run: copy ${{ github.workspace }}\wirego\wirego_plugin\* ${{ github.workspace }}\wireshark\plugins\epan\wirego\
+        run: copy ${{ github.workspace }}\wirego\wirego_bridge\* ${{ github.workspace }}\wireshark\plugins\epan\wirego\
       - name: Create build dir
         run: mkdir ${{ github.workspace }}\build
       - name: Debug dir

diff --git a/README.md b/README.md
@@ -1,42 +1,54 @@
-# Wirego
+# Wirego 
 
-A Wireshark plugin wrapper for golang
+|    |    |
+| -- | -- |
+| ![Wirego Logo](./doc/img/wirego_logo_small.png) |  A Wireshark plugin framework based on ZMQ, supporting Golang and hopefully more languages soon.
+|
 
 
-![Wirego Logo](./img/wirego_logo_small.png)
-
 ## Introduction
 
 Writing plugins for Wireshark in C/C++ can be opaque: the APIs are quite powerfull, but not really obvious to use. If you just want to develop a quick and dirty plugin you will spend more time trying to understand how things work instead of actually writing the core of your plugin.
 
 Another alternative is to use LUA, but first of all you need to know this language. So again, you'll spend more time trying to learn that new language than actually writing this quick and dirty plugin.
 
-Wirego is a plugin for Wireshark, written in C that actually loads a plugin written in Go language.
+Wirego is a composed of:
+
+  - a Wireshark plugin (wirego_bridge), written in C that will transmit all calls from Wireshark to a remote ZMQ endpoint
+  - A set of packages for several languages receiving those ZMQ calls and converting them to a simple API that you can use
+
+![screenshot](./doc/img/schema.png)
 
-You basically don't have to touch the Wirego plugin and you will be given a dummy empty golang plugin to start with.
+
+As a starter, a **golang** package is provided and more languages will come later.
 
 ![screenshot](./examples/minimal/screenshot.png)
 
-## Overview
+In all Wirego's code and documentations we will refer to:
+
+  - **Wirego bridge** : the Wireshark plugin, written in C (you won't have to touch this one)
+  - **Wirego package** : a package/class/bundle/sdk for a given language, used to make things easier on your side
+  - **Wirego remote** : the application that you will develop using the **Wirego package**
+
+## Overview (in Go)
 
 In order to setup Wirego, you will need follow 3 steps:
 
-  1. Install or build the Wirego plugin for Wireshark
-  2. Develop your own plugin, using the "wirego" Go package
-  3. Start Wireshark and tell Wirego where your plugin is
+  1. Install or build the **Wirego bridge plugin** for Wireshark
+  2. Develop your own plugin, using a **Wirego package**
+  3. Start Wireshark and tell the Wirego bridge where your ZMQ endpoint is
 
 You may use prebuilt binaries for **step 1**, those can be downloaded [here](https://github.com/quarkslab/wirego/releases).
-If prefer building the plugin (or if prebuilt binaries fails), refer to the following documentation [here](BUILD_WIREGO.md)
+If prefer building the plugin (or if prebuilt binaries fails), refer to the following documentation [here](./doc/BUILD_WIREGO.md)
 
 
-For **step 2**, you will basically just have to __import "wirego"__ and implement the following interface:
+The **step 2** will obviously depend on the language you're using. For Go you will basically just have to copy/paste the **main()** function from one of our examples and implement the following interface:
 
 ```golang
     // WiregoInterface is implemented by the actual wirego plugin
     type WiregoInterface interface {
       GetName() string
       GetFilter() string
-      Setup() error
       GetFields() []WiresharkField
       GetDetectionFilters() []DetectionFilterType
       GetDetectionHeuristicsParent() []string
@@ -45,28 +57,46 @@ For **step 2**, you will basically just have to __import "wirego"__ and implemen
     }
 ```
 
-Now it's time for **step 3**: [install the Wirego plugin and start Wireshark](RUNNING.md)!
+Now it's time for **step 3**: [install the Wirego plugin and start Wireshark](./doc/RUNNING.md)!
 
 ## Examples
 
 A few plugin examples are available :
 
-  - [Minimal](./examples/minimal/) ; a minimalistic example showing the basic usage of Wirego
+  - [Minimal](./examples/minimal/) : a minimalistic example showing the basic usage of Wirego
   - [Reolink Credentials light](./examples/reolinkcredslight/) : a lightweight version of a Reolink camera credentials parser
-  - [Reolink Credentials](./examples/reolinkcreds/) : a advanced version of a Reolink camera credentials parser
-
+  - [Reolink Credentials](./examples/reolinkcreds/) : an advanced version of a Reolink camera credentials parser
 
-## Next steps
+## Implementing a new language
 
-That project is still under development, many things needs to be improved.
-Here's a partial list:
-
-  - The fields type list is incomplete
-  - Support payload split into several packets
+If you plan to implement a package for a currently unsupported language, please take a look at the [Wirego ZMQ specifications](./doc/PROTOCOL.md).
 
 ## Additional notes
 
-When the path to your plugin in Go is modified, you will be required to restart Wireshark, here's why:
+When the ZMQ endpoint used by your **Wirego remote plugin** is modified, you will be required to restart Wireshark, here's why:
 
   - we need to setup everything (plugin name, fields..) during the proto_register_wirego call
-  - preferences values are only loaded during the proto_reg_handoff_wirego call, which is too late for us
+  - preferences values, hence the ZMQ endpoint, are only loaded afterwards during the proto_reg_handoff_wirego call
+
+## Changelog
+
+
+### Wirego 0.9 (18/12/2023)
+
+First public release of Wirego
+
+### Wirego 1.0 (26/03/2024)
+
+  - Plugins ABI updates to 1.1
+  - A detection heuristics function can now be defined
+  - Renamed DissectorFilter to DetectionFilters for more clarity
+
+### Wirego 2.0 (24/12/2024)
+
+Wirego 2.0 is a major update from Wirego 1.0.
+The communication between the Wireshark plugin and the end user plugin has been fully rewritten to allow more languages to be integrated later (Python, Rust...).
+
+  - Wirego's Wireshark plugin (wirego bridge) now uses ZMQ
+  - Golang package (wireshark remote) now receives commands from Wirego bridge
+  - Specification for ZMQ protocol (see doc/PROTOCOL.md)
+
diff --git a/BUILD_WIREGO.md → doc/BUILD_WIREGO.md b/BUILD_WIREGO.md → doc/BUILD_WIREGO.md
@@ -11,19 +11,16 @@ Use the following Dockerfile:
     FROM golang:latest
 
     # Install build tools and Wireshark dependencies
-    RUN DEBIAN_FRONTEND=noninteractive apt update && apt -y install git build-essential cmake libgcrypt-dev qt6-base-dev qt6-multimedia-dev qt6-tools-dev qt6-tools-dev-tools qt6-l10n-tools libqt6core5compat6-dev libpcap-dev libgcrypt20-dev libglib2.0-dev flex bison libpcre2-dev libnghttp2-dev libc-ares-dev libspeexdsp-dev
-    # Take a fresh version of Wireshark
-    RUN git clone https://gitlab.com/wireshark/wireshark.git
+    RUN DEBIAN_FRONTEND=noninteractive apt update && apt -y install git build-essential cmake libgcrypt-dev qt6-base-dev qt6-multimedia-dev qt6-tools-dev qt6-tools-dev-tools qt6-l10n-tools libqt6core5compat6-dev libpcap-dev libgcrypt20-dev libglib2.0-dev flex bison libpcre2-dev libnghttp2-dev libc-ares-dev libspeexdsp-dev libzmq5-dev
+    # Take a fresh version of Wireshark (you may want to change version to match your current install)
+    RUN git clone -b wireshark-4.4.2 https://gitlab.com/wireshark/wireshark.git
     # Take a fresh version of Wirego
     RUN git clone https://github.com/quarkslab/wirego.git
     # Link the wirego plugin folder to the Wireshark plugins source folder
     RUN ln -s /go/wirego/wirego_plugin /go/wireshark/plugins/epan/wirego
     # Build Wireshark and plugins
     RUN cd /go/wireshark && git checkout release-4.2 && mkdir build && cd build && cmake -DCUSTOM_PLUGIN_SRC_DIR=/go/wireshark/plugins/epan/wirego .. && make
 
-
-    # Build the example plugin
-    RUN cd /go/wirego/wirego/example/ && make
 ```
 
 
@@ -34,7 +31,6 @@ Build the Docker image using:
 Extract the built files from the image:
 
     id=$(docker create wiregobuild)
-    docker cp $id:/go/wirego/wirego/example/wirego_example.so - > wirego_example.so.tar
     docker cp $id:/go/wireshark/build/run/wireshark - > wireshark.so.tar
     docker cp $id:./wireshark/build/run/plugins/4.2/epan/wirego.so - > wirego.so.tar        
     docker rm -v $id
@@ -44,25 +40,33 @@ Extract the built files from the image:
 
 If you don't plan to use a pre-built version of the wirego plugin, you can built it manually.
 
+Install ZMQ library : [https://zeromq.org/download/](https://zeromq.org/download/)
+
+
 Clone Wireshark:
 
     git clone https://github.com/wireshark/wireshark.git
 
+Clone wirego:
+
+    git clone https://github.com/quarkslab/wirego.git
+
 Create a symlink from the Wireshark plugins folder to the "wirego_plugin"
 
-    ln -s <path_to>/wirego_plugin wireshark/plugins/epan/wirego
+    ln -s <path_to>/wirego/wirego_bridge wireshark/plugins/epan/wirego
 
 Edit Wireshark's main CMakelists.txt and add the following to PLUGIN_SRC_DIRS:
 
     plugins/epan/wirego
 
 Now build Wireshark (see README.xxx), but basically it's just:
 
+    cd wireshark
     mkdir build && cd build
     cmake ../
     make -j
 
-You may also just build the Wirego plugin (and now the full Wireshark program):
+You may also just build the Wirego plugin (and not the full Wireshark program):
 
     mkdir build && cd build
     cmake -DBUILD_wireshark=OFF -DCUSTOM_PLUGIN_SRC_DIR=../../wireshark/plugins/epan/wirego ..

diff --git a/doc/MIGRATING.md b/doc/MIGRATING.md
@@ -0,0 +1,52 @@
+# Migrating from Wirego v1 to v2
+
+The migration from Wirego v1 to Wirego v2 is quite easy.
+
+A few things have changed:
+
+  - You code will not be loaded anymore as a dynamic library
+  - The Setup inferface does not exist anymore
+  - You don't need the "init()" function anymore
+  - You need to implement the "main()" function
+  - You need to explicitely initialize the wirego package by using **New**
+
+
+## Step1: update
+
+Edit your main function, which used to be empty, as follow:
+
+    func main() {
+      var wge WiregoSample
+
+      wg, err := wirego.New("ipc:///tmp/wirego0", false, wge)
+      if err != nil {
+        fmt.Println(err)
+        return
+      }
+      wg.ResultsCacheEnable(false)
+
+      wg.Listen()
+    }
+
+You should edit on the above code snippet:
+
+  - var wge WiregoSample 
+  - wg.ResultsCacheEnable(false)
+
+## Step 2: cleanup
+
+Erase your init() and Setup functions.
+If needed, move contents to the main function.
+
+## Step 3: build
+
+Build using:
+
+    go build
+
+## Step 4: running
+
+When loading Wireshark, go to the Settings and update what used to be the plugin's library path, to the defined ZMQ endpoint (in the previous example: **"ipc:///tmp/wirego0"**).
+Restart Wireshark.
+
+