Merge pull request #16 from GreyDGL/log

fix login

.gitignore

@@ -6,7 +6,8 @@ __pycache__/
 config/chatgpt_config.py
 outputs/
 .idea
-log/
+logs/
+utils/logs/
 archive/
 
 # C extensions

README.md

@@ -1,6 +1,6 @@
 # PentestGPT
 
-**We're testing PentestGPT on HackTheBox**. Follow 
+**We're testing PentestGPT on HackTheBox**. More details will be released soon.
 
 ## Introduction
 **PentestGPT** is a penetration testing tool empowered by **ChatGPT**. It is designed to automate the penetration testing process. It is built on top of ChatGPT and operate in an interactive mode to guide penetration testers in both overall progress and specific operations.
@@ -43,6 +43,10 @@ https://user-images.githubusercontent.com/78410652/232327920-7318a0c4-bee0-4cb4-
    3. You may always use `TAB` to autocomplete the commands.
    4. When you're given a drop-down selection list, you can use cursor or arrow key to navigate the list. Press `ENTER` to select the item. Similarly, use <SHIFT + right arrow> to confirm selection.
 
+## Report
+1. After finishing the penetration testing, a report will be automatically generated in `logs` folder (if you quit with `quit` command).
+2. The report can be printed in a human-readable format by running `python3 utils/report_generator.py <log file>`. A sample report `sample_pentestGPT_log.txt` is also uploaded.
+
 
 ## Design Documentation
 The current design is mainly for web penetration testing

logs/sample_pentestGPT_log.txt

@@ -0,0 +1 @@
+{"user": [[1682155121.2103748, "I want to pentest 10.0.2.5, which is a machine from VulnHub"], [1682155158.6501088, "next"], [1682155218.291348, "quit"]], "pentestGPT": [[1682155206.655137, "Based on the analysis, the following tasks are recommended:Based on the provided Nmap scan results, I will update the task tree and remove the completed tasks. The updated task tree now looks like this:\n\n1. Reconnaissance\n  1.1. Passive Recon\n    1.1.1. OSINT\n  1.2. Active Recon\n    1.2.2. Service Enumeration\n      1.2.2.1. FTP Enumeration (Score: 3)\n      1.2.2.2. SSH Enumeration (Score: 2)\n      1.2.2.3. HTTP Enumeration (Score: 3)\n    1.2.3. Vulnerability Scanning (Score: 2)\n\n2. Exploitation\n  2.1. Web Exploitation (Score: 3)\n  2.2. Exploiting Services (Score: 2)\n  2.3. Privilege Escalation (Score: 3)\n\n3. Reporting\n  3.1. Document Vulnerabilities\n  3.2. Recommend Fixes\n\nBased on the updated task tree, I recommend the junior penetration tester to perform the following task:\n\n1.2.2.3. Enumerate the HTTP service on port 80 using tools like `nikto` and `gobuster` to identify potential vulnerabilities and gather information about the web application."], [1682155218.2919838, "Thank you for using PentestGPT!"]], "reasoning": [[1682155141.855479, "Based on the provided information, I have created an initial task tree. Here is the task tree:\n\n1. Reconnaissance\n  1.1. Passive Recon\n    1.1.1. OSINT\n  1.2. Active Recon\n    1.2.1. Network Scanning\n      1.2.1.1. Nmap Scan\n    1.2.2. Service Enumeration\n    1.2.3. Vulnerability Scanning\n\n2. Exploitation\n  2.1. Web Exploitation\n  2.2. Exploiting Services\n  2.3. Privilege Escalation\n\n3. Reporting\n  3.1. Document Vulnerabilities\n  3.2. Recommend Fixes\n\nNow, I'll suggest the next task for the junior penetration tester based on the task tree:\n\n1.2.1.1. Perform an Nmap scan on the target IP address (10.0.2.5) to identify open ports and running services."], [1682155206.651016, "Based on the provided Nmap scan results, I will update the task tree and remove the completed tasks. The updated task tree now looks like this:\n\n1. Reconnaissance\n  1.1. Passive Recon\n    1.1.1. OSINT\n  1.2. Active Recon\n    1.2.2. Service Enumeration\n      1.2.2.1. FTP Enumeration (Score: 3)\n      1.2.2.2. SSH Enumeration (Score: 2)\n      1.2.2.3. HTTP Enumeration (Score: 3)\n    1.2.3. Vulnerability Scanning (Score: 2)\n\n2. Exploitation\n  2.1. Web Exploitation (Score: 3)\n  2.2. Exploiting Services (Score: 2)\n  2.3. Privilege Escalation (Score: 3)\n\n3. Reporting\n  3.1. Document Vulnerabilities\n  3.2. Recommend Fixes\n\nBased on the updated task tree, I recommend the junior penetration tester to perform the following task:\n\n1.2.2.3. Enumerate the HTTP service on port 80 using tools like `nikto` and `gobuster` to identify potential vulnerabilities and gather information about the web application."]], "input_parsing": [[1682155179.923199, "I ran an Nmap scan on IP address 10.0.2.5 and found that it is up with a latency of 0.00043 seconds. Out of the 1000 scanned TCP ports, 997 were closed and reset. The following three ports are open: \n\n- Port 21: FTP\n- Port 22: SSH\n- Port 80: HTTP\n\nThe MAC address of the device is 08:00:27:3D:32:45, and the device type is a general-purpose computer running Linux 3.X or 4.X. The OS details indicate that it is running Linux 3.2 - 4.9. The network distance is one hop. \n\nOS detection was performed, and any incorrect results can be reported at https://nmap.org/submit/."]], "generation": [[1682155143.736203, "Sure, I can provide a first step for a penetration tester to take. \n\nThe first thing to do would be to perform a port scan on the target IP address to identify any open ports and services running on the target system. This can be done using the nmap tool with the following command:\n\n`nmap <target IP>`\n\nFor example, if the target IP address is 192.168.1.10, the command would be:\n\n`nmap 192.168.1.10`\n\nThis will provide information about which ports are open, which can be used to identify potential vulnerabilities or attack vectors."]], "exception": [[1682155143.741246, "PentestGPT suggests you to do the following: \nBased on the provided information, I have created an initial task tree. Here is the task tree:\n\n1. Reconnaissance\n  1.1. Passive Recon\n    1.1.1. OSINT\n  1.2. Active Recon\n    1.2.1. Network Scanning\n      1.2.1.1. Nmap Scan\n    1.2.2. Service Enumeration\n    1.2.3. Vulnerability Scanning\n\n2. Exploitation\n  2.1. Web Exploitation\n  2.2. Exploiting Services\n  2.3. Privilege Escalation\n\n3. Reporting\n  3.1. Document Vulnerabilities\n  3.2. Recommend Fixes\n\nNow, I'll suggest the next task for the junior penetration tester based on the task tree:\n\n1.2.1.1. Perform an Nmap scan on the target IP address (10.0.2.5) to identify open ports and running services."], [1682155143.7431931, "You may start with: \nSure, I can provide a first step for a penetration tester to take. \n\nThe first thing to do would be to perform a port scan on the target IP address to identify any open ports and services running on the target system. This can be done using the nmap tool with the following command:\n\n`nmap <target IP>`\n\nFor example, if the target IP address is 192.168.1.10, the command would be:\n\n`nmap 192.168.1.10`\n\nThis will provide information about which ports are open, which can be used to identify potential vulnerabilities or attack vectors."], [1682155177.259076, "Source: tool\nI performed nmap. Below is the result\nStarting Nmap 7.93 ( https://nmap.org ) at 2023-04-12 11:11 EDT\n- 3 - Nmap scan report for 10.0.2.5\n- 4 - Host is up (0.00043s latency).\n- 5 - Not shown: 997 closed tcp ports (reset)\n-6 - PORT STATE SERVICE\n- 7 - 21/tcp open ftp\n- 8 - 22/tcp open ssh\n- 9 - 80/tcp open http\n- 10 - MAC Address: 08:00:27:3D:32:45 (Oracle VirtualBox virtual NIC)\n- 11 - Device type: general purpose\n- 12 - Running: Linux 3.X|4.X\n- 13 - OS CPE: cpe:/o:linux:linux_kernel:3 cpe:/ o:linux:linux_kernel:4\n- 14 - OS details: Linux 3.2 - 4.9\n- 15 - Network Distance: 1 hop\n- 16 -\n- 17 - OS detection performed. Please report any incorrect results at https://nmap.org/submit/ .\n- 18 - Nmap done: 1 IP address (1 host up) scanned in 1.76 seconds"]]}

utils/chatgpt.py

@@ -132,6 +132,7 @@ def send_new_message(self, message, model=None):
         r = requests.post(
             url, headers=self.headers, json=data, proxies=self.proxies, stream=True
         )
+
         if r.status_code != 200:
             # wait for 20s
             logger.error(r.text)

utils/pentest_gpt.py

@@ -43,6 +43,7 @@ class pentestGPT:
     }
 
     def __init__(self):
+        self.log_dir = "logs"
         self.chatGPTAgent = ChatGPT(ChatGPTConfig())
         self.chatGPT4Agent = ChatGPT(ChatGPTConfig(model="gpt-4"))
         self.prompts = PentestGPTPrompt
@@ -379,8 +380,11 @@ def main(self):
         # TODO.
         # log the session.
         ## save self.history into a txt file based on timestamp
-        log_name = "pentestGPT_log_" + dt.now().strftime("%Y%m%d_%H%M%S") + ".json"
-        with open(log_name, "w") as f:
+        timestamp = time.time()
+        log_name = "pentestGPT_log_" + str(timestamp) + ".txt"
+        # save it in the logs folder
+        log_path = os.path.join(self.log_dir, log_name)
+        with open(log_path, "w") as f:
             json.dump(self.history, f)
 
         # clear the sessions

utils/report_generator.py

@@ -0,0 +1,42 @@
+# a quick report generation script that converts the saved logs file into a pdf.
+import json, os, sys
+import datetime, time
+
+
+def main(file_name):
+    # load the file into json
+    with open(file_name, "r") as f:
+        logs = json.load(f)
+    user_inputs = logs["user"]
+    bot_responses = logs["pentestGPT"]
+    # merge the two list into one. Sort based on the first element (timestamp in str), and add a flag to show source
+    merged_list = []
+    for user_input in user_inputs:
+        merged_list.append([user_input[0], user_input[1], "user"])
+    for bot_response in bot_responses:
+        merged_list.append([bot_response[0], bot_response[1], "pentestGPT"])
+    merged_list.sort(key=lambda x: x[0])
+
+    # now print the conversation
+    output = ""
+    for element in merged_list:
+        # convert the timestamp to a human readable format
+        timestamp = datetime.datetime.fromtimestamp(int(element[0])).strftime(
+            "%Y-%m-%d %H:%M:%S"
+        )
+        output += f"{timestamp} [{element[2]}]: {element[1]}\n"
+        # add an additional line break if the element is from bot
+        if element[2] == "pentestGPT":
+            output += "----------------------------------------\n\n"
+    # print the output
+    print("Conversation log: ")
+
+    print(output)
+
+if __name__ == "__main__":
+    # default filename = "../logs/sample_pentestGPT_log.txt"
+    if len(sys.argv) == 1:
+        file_name = "logs/sample_pentestGPT_log.txt"
+    else:
+        file_name = sys.argv[1]
+    main(file_name)