Merge pull request #10 from PaperMtn/feature/custom-list-enhance

Version 3.1.0 Release

CHANGELOG.md

@@ -1,3 +1,21 @@
+## [3.1.0] - 2024-08-13
+### Added
+- Added new functionality to enhance the custom passwords passed to lil-pwny
+  - Lil Pwny can now take the custom password list and create a number of variations of each password in the list:
+    - Passwords with common 'leetspeak' substitutions (e.g. `P@ssw0rd`)
+    - Uppercase versions of the password, and uppercase first characters (e.g. `PASSWORD`, `Password`)
+    - Passwords with common special characters appended or prepended (e.g. `password!`, `!password`)
+    - Passwords padded with common alphanumeric characters, special characters and repetitions of themselves to make them meet a given minimum length (e.g. `password123!`, `!passwordabc`, `passwordpassword`)
+    - Passwords with dates appended starting from the year 1950 up to 10 years from today's date (e.g. `password1950`, `password2034`)
+  - To give an idea, a password list of 100 custom passwords generates 49848660 variations
+- Logging now includes the plaintext password for custom password list matches
+  - This is useful for identifying the password that was found in the custom password list
+  - These are redacted if the `--obfuscate` flag is used
+
+### Changed
+- `--debug` option switched to `--verbose` to more accurately describe output
+  - Some logging output moved from INFO level to DEBUG (displayed when `--verbose` is passed)
+
 ## [3.0.1] - 2024-07-22
 ### Added
 - Updated logging

README.md

@@ -10,13 +10,25 @@ Fast offline auditing of Active Directory passwords using Python.
 ## About Lil Pwny
 Lil Pwny is a Python application to perform an offline audit of NTLM hashes of users' passwords, recovered from Active Directory, against known compromised passwords from Have I Been Pwned. Results will be output in JSON format containing the username, matching hash (can be obfuscated), and how many times the matching password has been seen in HIBP
 
-There are also additional features:
-- Ability to provide a list of your own custom passwords to check AD users against. This allows you to check user passwords against passwords relevant to your organisation that you suspect people might be using. These are NTLM hashed, and AD hashes are then compared with this as well as the HIBP hashes.
-- Return a list of accounts using the same passwords. Useful for finding users using the same password for their administrative and standard accounts.
-- Obfuscate hashes in output, for if you don't want to handle or store live user NTLM hashes.
-
 More information about Lil Pwny can be found [on my blog](https://papermtn.co.uk/category/tools/lil-pwny/)
 
+## Features
+
+- **Custom Password Auditing**: Ability to provide a list of your own custom passwords to check AD users against. This allows you to check user passwords against passwords relevant to your organisation that you suspect people might be using. 
+  - Pass a .txt file with the plaintext passwords you want to search for, these are then NTLM hashed and AD hashes are then compared with this as well as the HIBP hashes.
+- **Detect Duplicates**: Return a list of accounts using the same passwords. Useful for finding users using the same password for their administrative and standard accounts.
+- **Obfuscated Output**: Obfuscate hashes in output, for if you don't want to handle or store live user NTLM hashes.
+
+### Custom Password List Enhancement
+Lil Pwny provides the functionality to enhance your custom password list by adding commonly used variants of your custom passwords. These include:
+- Passwords with common 'leetspeak' substitutions (e.g. `P@ssw0rd`)
+- Uppercase versions of the password, and uppercase first characters (e.g. `PASSWORD`, `Password`)
+- Passwords with common special characters appended or prepended (e.g. `password!`, `!password`)
+- Passwords padded with common alphanumeric characters, special characters and repetitions of themselves to make them meet a given minimum length (e.g. `password123!`, `!passwordabc`, `passwordpassword`)
+  - You pass your desired minimum password length to Lil Pwny when selecting the custom list enhancement option
+- Passwords with dates appended starting from the year 1950 up to 10 years from today's date (e.g. `password1950`, `password2034`)
+
+A custom password list of 100 plaintext passwords generates 49848660 variations.
 ## Resources
 This application has been developed to make the most of multiprocessing in Python, with the aim of it working as fast as possible on consumer level hardware.
 
@@ -26,10 +38,14 @@ Because it uses multiprocessing, the more cores you have available, the faster L
     - 12 logical cores - 0:04:28.579201
 
 ## Output
-Lil Pwny will output results as either to stdout or JSON:
+Lil Pwny will output results as either to stdout:
+
+<img src="./images/stdout-screenshot.png" width="675">
+
+or JSON:
 
 ```json
-{"localtime": "2021-00-00 00:00:00,000", "level": "NOTIFY", "source": "Lil Pwny", "match_type": "hibp", "detection_data": {"username": "RICKON.STARK", "hash": "0C02C50B2B08F2979DFDE12EDA472FC1", "matches_in_hibp": "24230577", "obfuscated": "True"}}
+{"localtime": "2021-00-00 00:00:00,000", "level": "NOTIFY", "source": "Lil Pwny", "match_type": "hibp", "detection_data": {"username": "RICKON.STARK", "hash": "32ED87BDB5FDC5E9CBA88547376818D4", "matches_in_hibp": "24230577", "obfuscated": "True"}}
 ```
 You can redirect the JSON output of Lil Pwny to file:
 ```commandline
@@ -48,25 +64,27 @@ pip install lil-pwny
 Lil-pwny will be installed as a global command, use as follows:
 
 ```
-usage: lil-pwny [-h] -hibp HIBP [--version] [-c CUSTOM] -ad AD_HASHES [-d] [-output {file,stdout,json}] [-o] [--debug]
+usage: lil-pwny [-h] -hibp HIBP [-v] [-c CUSTOM] [-custom-enhance CUSTOM_ENHANCE] -ad AD_HASHES [-d] [-output {file,stdout,json}] [-o] [--verbose]
 
 Fast offline auditing of Active Directory passwords using Python
 
 options:
   -h, --help            show this help message and exit
   -hibp HIBP, --hibp HIBP
                         The .txt file containing HIBP NTLM hashes
-  --version             show program's version number and exit
+  -v, --version         show program's version number and exit
   -c CUSTOM, --custom CUSTOM
                         .txt file containing additional custom passwords to check for
+  -custom-enhance CUSTOM_ENHANCE, --custom-enhance CUSTOM_ENHANCE
+                        generate an enhanced custom password list based on the provided custom password list. Must be used with -c/--custom flag. The enhanced list will stored in memory and not
+                        written to disk. Provide the minimum length of the passwords you want. Default is 8
   -ad AD_HASHES, --ad-hashes AD_HASHES
                         The .txt file containing NTLM hashes from AD users
   -d, --duplicates      Output a list of duplicate password users
   -output {file,stdout,json}, --output {file,stdout,json}
                         Where to send results
   -o, --obfuscate       Obfuscate hashes from discovered matches by hashing with a random salt
-  --debug               Turn on debug level logging
-
+  --verbose             Turn on verbose logging
 
 ```
 
@@ -103,5 +121,12 @@ Get-ADDBAccount -All -DBPath '.\Active Directory\ntds.dit' -BootKey $bootKey | F
 ### Step 3: Download the latest HIBP hash file
 The file can be downloaded from the HIBP API using a .net utility  [here](https://github.com/HaveIBeenPwned/PwnedPasswordsDownloader)
 
+### Optional Step: Filter unwanted AD accounts
+The PowerShell script in the [scripts](./scripts/Filter-ADUsers) directory can be used to remove unwanted accounts from the IFM output before processing. These include:
+
+- Disabled accounts
+- Computer accounts
+
+
 ## Resources
 - [ntdsutil & IFM](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-r2-and-2012/cc732530(v=ws.11))

pyproject.toml

@@ -1,13 +1,13 @@
 [tool.poetry]
 name = "lil-pwny"
-version = "3.0.1"
+version = "3.1.0"
 description = "Fast offline auditing of Active Directory passwords using Python and multiprocessing"
 authors = ["PaperMtn <papermtn@protonmail.com>"]
 license = "GPL-3.0"
 readme = "README.md"
 
 [tool.poetry.dependencies]
-python = "^3.12"
+python = ">=3.11"
 colorama = "^0.4.6"
 pycryptodome = "^3.20.0"
 charset-normalizer = "^3.3.2"

scripts/Filter-ADUsers/Filter-ADUsers.ps1

@@ -0,0 +1,54 @@
+<#
+.DESCRIPTION
+    This script Filters username and NTLM hash pairs from the given IFM output. It excludes entries that are for disabled users or computer accounts by checking against Active Directory using the ActiveDirectory module. Users are prompted to select a file at runtime.
+.PARAMETER InputFile
+    The input file selected by the user, containing username:hash pairs from the AD IFM dump.
+.OUTPUTS
+    A file named 'filtered_ad_hashes.txt' containing the filtered username:hash pairs from Active Directory output.
+.EXAMPLE
+    .\Filter-ADUsers.ps1
+    This command runs the script and prompts the user to select an input file for processing.
+#>
+
+Import-Module ActiveDirectory
+Add-Type -AssemblyName System.Windows.Forms
+
+# Create an OpenFileDialog to prompt the user for a file
+$OpenFileDialog = New-Object System.Windows.Forms.OpenFileDialog
+$OpenFileDialog.InitialDirectory = (Get-Location).Path
+$OpenFileDialog.Filter = "Text files (*.txt)|*.txt|All files (*.*)|*.*"
+$OpenFileDialog.Title = "Select the hashes file"
+
+# Show the dialog and get the selected file
+$OpenFileDialog.ShowDialog() | Out-Null
+$selectedFilePath = $OpenFileDialog.FileName
+
+# Check if a file was selected
+if (-not [string]::IsNullOrEmpty($selectedFilePath)) {
+    # Initialize an array to store the output
+    $outputArray = @()
+
+    # Read the content of the selected file and process each line
+    Get-Content $selectedFilePath | ForEach-Object {
+        # Split each line into username and hash
+        $username = $_.split(":")[0]
+        $hash = $_.split(":")[1]
+
+        # Check if the username does not end with a dollar sign - a computer account
+        if ($username -notmatch '\$$') {
+            if ($username -notmatch "[^a-zA-Z0-9.]") {
+                $adUser = Get-ADUser -Filter "SamAccountName -eq '$username'"
+                # Check if the user account is enabled
+                if ($adUser.Enabled -eq $true) {
+                    $outputArray += "$($username):$($hash)"
+                }
+            }
+        }
+    }
+
+    $outputArray | Out-File -FilePath .\filtered_ad_hashes.txt
+
+    Write-Output "Filtering complete. The output has been saved to 'filtered_ad_hashes.txt'."
+} else {
+    Write-Output "No file was selected. Exiting script."
+}

scripts/Filter-ADUsers/README.md

@@ -0,0 +1,8 @@
+# Filter Active Directory output
+You can use the script Filter-ADUsers.ps1 to filter the following out from the IFM export from Active Directory:
+- Disabled accounts
+- Computer accounts
+
+This saves you from processing accounts that aren’t useful.
+
+**Note**: You will need to have Remote Server Administrative Tools (RSAT) added from optional features in Windows to use the `ActiveDirectory` PowerShell module.