Merge pull request #139 from usdAG/develop

Release v1.3.1

Author: fhaag95

.github/workflows/develop.yml

@@ -23,10 +23,10 @@ jobs:
 
     - uses: actions/checkout@v2
 
-    - name: Set up JDK 1.8
+    - name: Set up JDK 17
       uses: actions/setup-java@v1
       with:
-        java-version: 1.8
+        java-version: 17
 
     - name: Cache local Maven repository
       uses: actions/cache@v2

.github/workflows/master.yml

@@ -23,10 +23,10 @@ jobs:
 
     - uses: actions/checkout@v2
 
-    - name: Set up JDK 1.8
+    - name: Set up JDK 17
       uses: actions/setup-java@v1
       with:
-        java-version: 1.8
+        java-version: 17
 
     - name: Cache local Maven repository
       uses: actions/cache@v2

.gitignore

@@ -11,3 +11,4 @@ buildNumber.properties
 .classpath
 .project
 .settings/
+*/launch.json

INTRODUCTION.md

@@ -0,0 +1,92 @@
+# CSTC Introduction
+This document serves as a written introduction to the Cyber Security Transformation Chef or in short: CSTC. It starts by giving an overview of the general UI of the tool and after that walking through a demonstrative use-case explaining the core functionalities along the way.
+
+## UI Overview
+
+### Main Panel
+
+<img src="media/introduction/fig01-overview.png" width="75%" height="75%">
+
+The UI can be divided into three functional areas.<br>
+On the far left **(1)** is the area for selecting operations, grouped categorically. These are added to the recipe in the appropriate lane via click-and-drag. There is a search bar above the operations tree for quick access.<br>
+The recipe panel is located in the middle **(2)**. This is made up of up to 10 lanes, whereby the operations are applied in the order from top to bottom and left to right. Each lane works anew on the input sent to the CSTC. This makes it possible to carry out several transformations on the same input.<br>
+Finally, the selection area on the right **(3)** provides an overview of the initial development of a recipe and is used for debugging a recipe. The input to be worked with is shown in the upper area, and the lower area shows the result after the recipe has been applied.
+
+
+### Filter
+
+<img src="media/introduction/fig02-filter.png" width="75%" height="75%">
+
+The CSTC enables HTTP requests and responses to be changed automatically according to the given recipe. The tabs for which the recipes are to be applied are selected using the ```Filter``` button at the top middle of the recipe panel. This opens the pop-up window for selection.
+
+
+### Different Recipes
+
+<img src="media/introduction/fig03-different_recipes.png" width="75%" height="75%">
+
+The CSTC enables parallel work with HTTP requests and responses. Exactly one recipe can be created for each of them. Which recipe you are currently working on is controlled via the tab selection in the top left corner. In the tab ```Outgoing Requests``` you work on the HTTP requests, in ```Incoming Responses``` you work analogously on the HTTP responses. The third tab ```Formatting``` offers space to work with data independently of requests and responses and has no effect on regarding automatic transformation of requests/responses. It can be used to test recipes or perform static transformations comparable to the GCHQ CyberChef.
+
+
+## Example 1 - Response
+
+We will now look at two examples using a demo application. A detailed video demonstration of the CSTC can be found [here](https://www.youtube.com/watch?v=6fjW4iXj5cg).
+
+<img src="media/introduction/fig04-send_to_incoming.png" width="75%" heigth="75%">
+
+In this first example we see a HTTP request and its response in the Repeater tab. The body of the response is encoded and to create a matching recipe with the CSTC, we send the response to the ```Incoming``` tab.<br>
+Note here that the menu for sending the HTTP request to the CSTC can also be called up in the Proxy tab and, above all, in the HTTP history.
+
+<img src="media/introduction/fig05-example_1_response.png" width="75%" height="75%">
+
+Now the appropriate recipe must be created. In this case we use two lanes **(1)**: In the first, we extract the body of the HTTP response, decode it and store it in a variable named ```body```. In the second lane, we replace the body of the original HTTP response with the body we have manipulated and stored in the variable. We also add a suitable Content-Type Header so that Burp knows how to display the data in pretty print. We see the result on the right **(2)**. We finally instruct the CSTC to apply this recipe to all incoming responses in the Repeater tab **(3)**.
+
+<img src="media/introduction/fig06-example_1_poc.png" width="75%" height="75%">
+
+If we now resend our request in the Repeater tab, we see that the recipe is working.
+
+
+## Example 2 - Request
+
+<img src="media/introduction/fig07-example_2.png" width="75%" height="75%">
+
+For a second example, let's take a look at this HTTP POST request. We have three POST parameters and want to test the first parameter for SQL Injection. However, every time the value is changed, the API responds with an error message that the checksum is incorrect. In this case we found out that the values of the first two parameters are concatenated and then the SHA1 value is calculated of the resulting string. The result is cross-checked with the value of the integrity parameter. With the help of the CSTC, this scheme can be automatically applied to all outgoing requests and the testing process is greatly simplified.
+
+<img src="media/introduction/fig08-send_to_outgoing.png" width="75%" height="75%">
+
+As before, we send the data to the CSTC to be able to work with it. This time we work with the HTTP request, so we send it to the ```Outgoing``` tab.
+
+<img src="media/introduction/fig09-load_recipe.png" width="75%" height="75%">
+
+At this point, another feature of the CSTC can be demonstrated. Created recipes can be saved in the local file system and reloaded if necessary. Here, selecting ```Load``` **(1)** opens a pop-up and the saved recipe can be selected **(2)**.
+
+<img src="media/introduction/fig10-example_1_recipe.png" width="75%" heigth="75%">
+
+As you can see in the overview on the right, the value of the integrity parameter is now recalculated dynamically depending on the values of the request.
+
+<img src="media/introduction/fig11-example_2_filter.png" width="75%" height="75%">
+
+We now click on ```Filter``` again to select that the recipe should be applied to outgoing requests in the Repeater tab.
+
+<img src="media/introduction/fig12-example_2_poc.png" width="75%" height="75%">
+
+When resending the request in the Repeater tab, we receive an Internal Server Error, which means that the checksum test was successful and we can start testing the POST parameters.
+
+<img src="media/introduction/fig13-example_2_sqli.png" width="75%" height="75%">
+
+With an appropriately adapted payload, we can now verify and exploit a SQL injection vulnerability in this API endpoint.
+
+### Automation with the help of the CSTC
+
+Suppose we wanted to test the POST parameter using the Burp Scanner. Without adapting the integrity POST parameter, it is almost impossible to carry out a meaningful test. It is useful here that CSTC recipes can also be used for the Scanner.
+
+<img src="media/introduction/fig14-scanner_filter.png" width="75%" height="75%">
+
+First, we activate the use of the CSTC recipe for the Scanner.
+
+<img src="media/introduction/fig15-intruder.png" width="75%" height="75%">
+
+In the Intruder tab we now mark the parameter **(1)** to be tested and select the displayed menu item **(2)**. After selecting a suitable scan configuration, the scan can be started.
+
+<img src="media/introduction/fig16-scan_result.png" width="75%" height="75%">
+
+Using the CSTC recipe for outgoing requests, the Burp Scanner was able to confirm the SQLi as the CSTC transforms all requests containing payloads dynamically by applying the defined recipe shown above. This shows that the good integration of the CSTC can also be chained with other Extensions or builtin functions of Burp Suite.

README.md

@@ -83,12 +83,6 @@ Take a look at our basic tutorial on [YouTube](https://www.youtube.com/watch?v=B
 **UPDATE:** Due to some incompatibility issues when installing *CSTC* via *BApp Store*, we had to switch to a new variable prefix.
 Variables from other *lanes* have now to be prefixed by ``$`` e.g. like ``$Outgoing_step1``.
 
-
-## Known Issues
-
-Unfortunately, the GUI of some *CSTC Operations* does not really work well together with the **dark theme** of *Burp Suite*. Therefore,
-we recommend to use a **light theme** for the best user experience.
-
 ## Feedback
 
 We gladly appreciate all feedback, bug reports and feature requests.

media/introduction/fig01-overview.png

@@ -4,7 +4,7 @@
 	<modelVersion>4.0.0</modelVersion>
 	<groupId>de.usd.CSTC</groupId>
 	<artifactId>CSTC</artifactId>
-	<version>1.3.0</version>
+	<version>1.3.1</version>
 	<name>CSTC</name>
 	<description>CSTC</description>
 
@@ -17,13 +17,13 @@
 		<dependency>
 			<groupId>com.jayway.jsonpath</groupId>
 			<artifactId>json-path</artifactId>
-			<version>2.7.0</version>
+			<version>2.9.0</version>
 		</dependency>
 
 		<dependency>
-			<groupId>net.portswigger.burp.extender</groupId>
-			<artifactId>burp-extender-api</artifactId>
-			<version>2.3</version>
+			<groupId>net.portswigger.burp.extensions</groupId>
+			<artifactId>montoya-api</artifactId>
+			<version>2023.12.1</version>
 		</dependency>
 
 		<dependency>
@@ -35,19 +35,25 @@
 		<dependency>
 			<groupId>com.fasterxml.jackson.core</groupId>
 			<artifactId>jackson-core</artifactId>
-			<version>2.14.2</version>
+			<version>2.17.1</version>
 		</dependency>
 
 		<dependency>
 			<groupId>com.fasterxml.jackson.core</groupId>
 			<artifactId>jackson-databind</artifactId>
-			<version>2.14.2</version>
+			<version>2.17.1</version>
 		</dependency>
 
 		<dependency>
 			<groupId>org.apache.commons</groupId>
 			<artifactId>commons-text</artifactId>
-			<version>1.10.0</version>
+			<version>1.12.0</version>
+		</dependency>
+		
+		<dependency>
+		    <groupId>com.auth0</groupId>
+		    <artifactId>java-jwt</artifactId>
+		    <version>4.4.0</version>
 		</dependency>
 
         <dependency>
@@ -56,6 +62,12 @@
           <version>4.13.2</version>
           <scope>test</scope>
         </dependency>
+
+		<dependency>
+    		<groupId>org.javatuples</groupId>
+    		<artifactId>javatuples</artifactId>
+    		<version>1.2</version>
+		</dependency>
 
 	</dependencies>
 
@@ -80,17 +92,17 @@
 
 			<plugin>
 				<artifactId>maven-compiler-plugin</artifactId>
-				<version>3.11.0</version>
+				<version>3.13.0</version>
 				<configuration>
-					<source>1.8</source>
-					<target>1.8</target>
+					<source>17</source>
+					<target>17</target>
 				</configuration>
 			</plugin>
 
 			<plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-surefire-plugin</artifactId>
-                <version>3.0.0</version>
+                <version>3.2.5</version>
                 <configuration>
         		  <trimStackTrace>false</trimStackTrace>
 				  <redirectTestOutputToFile>true</redirectTestOutputToFile>

media/introduction/fig02-filter.png

@@ -1,114 +1,59 @@
 package burp;
 
-import java.awt.Component;
-import java.awt.event.ActionEvent;
-import java.awt.event.ActionListener;
-import java.util.ArrayList;
-import java.util.List;
+import com.fasterxml.jackson.databind.ObjectMapper;
 
-import javax.swing.JMenuItem;
-
-import de.usd.cstchef.view.FormatTab;
-import de.usd.cstchef.view.RecipePanel;
+import burp.api.montoya.BurpExtension;
+import burp.api.montoya.MontoyaApi;
+import burp.api.montoya.core.BurpSuiteEdition;
+import burp.api.montoya.persistence.PersistedObject;
+import de.usd.cstchef.view.RequestFilterDialog;
 import de.usd.cstchef.view.View;
+import de.usd.cstchef.view.filter.FilterState;
+import de.usd.cstchef.view.filter.FilterState.BurpOperation;
 
-public class BurpExtender implements IBurpExtender, ITab, IMessageEditorTabFactory, IHttpListener, IContextMenuFactory {
+public class BurpExtender implements BurpExtension {
 
     private final String extensionName = "CSTC";
-    private IBurpExtenderCallbacks callbacks;
     private View view;
 
     @Override
-    public void registerExtenderCallbacks(final IBurpExtenderCallbacks callbacks) {
-        this.callbacks = callbacks;
-        Logger.getInstance().init(callbacks.getStdout(), callbacks.getStderr());
-        BurpUtils.getInstance().init(callbacks);
-
-        callbacks.setExtensionName(this.extensionName);
-        callbacks.addSuiteTab(this);
-        callbacks.registerHttpListener(this);
-        callbacks.registerContextMenuFactory(this);
-        callbacks.registerMessageEditorTabFactory(this);
-    }
-
-
-    @Override
-    public String getTabCaption() {
-        return this.extensionName;
-    }
-
-    @Override
-    public Component getUiComponent() {
+    public void initialize(MontoyaApi api) {
+        BurpUtils.getInstance().init(api);
         this.view = new View();
-        return this.view;
-    }
-
-    @Override
-    public void processHttpMessage(int toolFlag, boolean messageIsRequest, IHttpRequestResponse messageInfo) {
-        if (messageIsRequest && view.getOutgoingRecipePanel().shouldProcess(toolFlag)) {
-            byte[] request = messageInfo.getRequest();
-            byte[] modifiedRequest = view.getOutgoingRecipePanel().bake(request);
-            Logger.getInstance().log("modified request: \n" + new String(modifiedRequest));
-            messageInfo.setRequest(modifiedRequest);
-        } else if (view.getIncomingRecipePanel().shouldProcess(toolFlag)) {
-            byte[] response = messageInfo.getResponse();
-            byte[] modifiedResponse = view.getIncomingRecipePanel().bake(response);
-            messageInfo.setResponse(modifiedResponse);
-            Logger.getInstance().log("modified response: \n" + new String(modifiedResponse));
+        BurpUtils.getInstance().setView(view);
+        api.extension().setName(extensionName);
+        api.userInterface().registerContextMenuItemsProvider(new CstcContextMenuItemsProvider(api, view));
+        api.http().registerHttpHandler(new CstcHttpHandler(view));
+        api.userInterface().registerSuiteTab(extensionName, view);
+        api.userInterface().registerHttpRequestEditorProvider(new MyHttpRequestEditorProvider(view));
+        api.userInterface().registerHttpResponseEditorProvider(new MyHttpResponseEditorProvider(view));
+
+        if (!api.burpSuite().version().edition().equals(BurpSuiteEdition.COMMUNITY_EDITION)) {
+            PersistedObject persistence = api.persistence().extensionData();
+            restoreFilterState(persistence);
+            restoreRecipe(persistence);
         }
+        view.updateInactiveWarnings();
     }
 
-    @Override
-    public List<JMenuItem> createMenuItems(IContextMenuInvocation invoc) {
-
-        List<JMenuItem> menuItems = new ArrayList<>();
-        JMenuItem incomingMenu = new JMenuItem("Send to CSTC (Incoming)");
-        JMenuItem outgoingMenu = new JMenuItem("Send to CSTC (Outgoing)");
-        JMenuItem incomingFormatMenu = new JMenuItem("Send to CSTC (Formating)");
-
-        menuItems.add(incomingMenu);
-        menuItems.add(outgoingMenu);
-        menuItems.add(incomingFormatMenu);
-
-        incomingMenu.addActionListener(new ActionListener() {
-            @Override
-            public void actionPerformed(ActionEvent e) {
-                IHttpRequestResponse[] msgs = invoc.getSelectedMessages();
-                if (msgs != null && msgs.length > 0) {
-                    view.getIncomingRecipePanel().setInput(msgs[0]);
-                }
-            }
-        });
-
-        outgoingMenu.addActionListener(new ActionListener() {
-            @Override
-            public void actionPerformed(ActionEvent e) {
-                IHttpRequestResponse[] msgs = invoc.getSelectedMessages();
-                if (msgs != null && msgs.length > 0) {
-                    view.getOutgoingRecipePanel().setInput(msgs[0]);
-                }
-
-            }
-        });
-
-        incomingFormatMenu.addActionListener(new ActionListener() {
-            @Override
-            public void actionPerformed(ActionEvent e) {
-                IHttpRequestResponse[] msgs = invoc.getSelectedMessages();
-                if (msgs != null && msgs.length > 0) {
-                    view.getFormatRecipePanel().setInput(msgs[0]);
-                }
-            }
-        });
-
-        return menuItems;
+    private void restoreRecipe(PersistedObject persistence) {
+        try {
+            this.view.getFormatRecipePanel().restoreState(persistence.getString(BurpOperation.FORMAT + "Recipe"));
+            this.view.getIncomingRecipePanel().restoreState(persistence.getString(BurpOperation.INCOMING + "Recipe"));
+            this.view.getOutgoingRecipePanel().restoreState(persistence.getString(BurpOperation.OUTGOING + "Recipe"));
+        } catch (Exception e) {
+            Logger.getInstance().log(
+                    "Could not restore the recipe for one or multiple panels. If this is the first time using CSTC in a project, you can ignore this message.");
+        }
     }
 
-    @Override
-    public IMessageEditorTab createNewInstance(IMessageEditorController controller, boolean editable) {
-        RecipePanel requestFormatPanel = this.view.getOutgoingRecipePanel();
-        // TODO do we need the format panel or do we want to use the incoming recipe?
-        RecipePanel responseFormatPanel = this.view.getFormatRecipePanel();
-        return new FormatTab(requestFormatPanel, responseFormatPanel, editable);
+    private void restoreFilterState(PersistedObject persistence) {
+        try {
+            BurpUtils.getInstance().setFilterState(new ObjectMapper().readValue(persistence.getString("FilterState"), FilterState.class));
+            RequestFilterDialog.getInstance().updateFilterSettings();
+        } catch (Exception e) {
+            Logger.getInstance().log(
+                    "Could not restore the filter state. If this is the first time using CSTC in a project, you can ignore this message. " + e.getMessage());
+        }
     }
 }