Initial commit (#1)

* Initial commit

* rev 1

* remove uneeded graphql calls

* Update .gitignore

* Update cli.py

* fixing queries

* refactor, j1 python module

* Delete .idea directory

* dashboard

* added documentation

* Update j1api.py

* doc update

* bug fix and new target query

* pypi installer

* update readme with pip install

.gitignore

@@ -0,0 +1,8 @@
+*.pyc
+.idea/
+reports/*
+report_map.json
+.DS_Store
+build/
+dist/
+j1nuclei.egg-info/

MANIFEST.in

@@ -0,0 +1,4 @@
+include MANIFEST.in
+include LICENSE
+include README.md
+include j1nuclei/targets_discovery.json

README.md

@@ -1 +1,80 @@
-# j1nuclei
+<img src="jupiterone_nuclei.png" width="75%">
+<br>
+
+## About
+J1Nuclei is a CLI tool demonstrating how JupiterOne platform can automate and learn from other tools. 
+It automates everyday security tasks of scanning endpoints for vulnerabilities. Once scans are complete, the tool returns findings to our JupiterOne knowledge graph. Results can be reviewed, prioritized, and measured using Jupiterone console and insight dashboards.
+
+### Quickstart
+
+The tool can be installed by simply cloning the repository and starting the module.
+
+1. [Get Started](https://info.jupiterone.com/start-account) - If you're not already using JupiterOne, it's free (no credit card).
+2. Install nuclei<br>
+``pip install nuclei``
+3. Install j1nuclei<br>
+   * PIP ``pip install j1nuclei``
+   * From source ``git clone git@github.com:JupiterOne/j1nuclei.git``
+3. Get JupiterOne API token<br>
+Follow instructions from [Create User and Account API](https://community.askj1.com/kb/articles/785-creating-user-and-account-api-keys) Keys kb article.
+<br>
+4. Export your api key to the environment variable ``J1_API_TOKEN``<br>
+``export J1_API_TOKEN="<your key>"``
+5. Launch j1nuclei from console or terminal<br>
+<img src="j1nuclei-cli.png" width="70%" height="75%">
+
+## Exploring Findings
+Findings are mapped back into our graph using the following schema<br>
+<img src="schema.png">
+### 1. JupiterOne Query Language (J1QL)
+More information about J1QL is available from [Introduction to JupiterOne Query Language](https://community.askj1.com/kb/articles/980-introduction-to-jupiterone-query-language-j1ql)
+The J1QL and knowledge graph can answer many questions, here's a few from the data set produced by J1Nuclei
+
+#### How many nuclei issues do I have?
+```
+FIND nuclei_finding as f
+RETURN count(f) as value
+```
+
+#### How many my critical assets in production are affected?
+```
+FIND *
+WITH tag.Production = true AND classification = 'critical' AS asset
+THAT HAS >> nuclei_finding
+RETURN COUNT(asset)
+```
+#### How many endpoints are affected?
+```
+FIND UNIQUE * as asset
+THAT HAS >> nuclei_finding
+RETURN count(asset) as value
+```
+
+#### Criticality of the issues?
+```
+FIND nuclei_finding as f
+WHERE f._type = 'nuclei_finding'
+RETURN f.severity as x, count(f) as y
+```
+
+#### What are my issues (graph view)?
+```
+FIND *
+THAT HAS >> nuclei_finding
+THAT IS >> nuclei_vulnerability
+RETURN TREE
+```
+
+### 2. Insight Dashboard
+You can also create dashboards using our console Insights. For starters, you can use the one we provided as part
+of this tool [nuclei_portal_schema.json](dashboard_nuclei_portal.json). Steps to create, edit, and upload your own
+dashboard are available from [Getting started with insights-dashboards](https://community.askj1.com/kb/articles/812-getting-started-with-insights-dashboards).
+We also shared many dashboards in our open-source repository from [https://github.com/JupiterOne/insights-dashboards](https://github.com/JupiterOne/insights-dashboards).
+
+<img src="insight-dashboard.png" width="60%" height="60%">
+
+## Customizing target discovery
+Because getting a comprehensive view may require several queries, j1nuclei use a JSON file [target_query.json](j1nuclei/targets_discovery.json)
+to define all queries to run. The file is populated with common queries by default and is extensible with any J1QL queries.
+For more information on our J1QL language is available from our [support site]("https://community.askj1.com/kb/articles/980-introduction-to-jupiterone-query-language-j1ql") 
+and other questions implementation is available from [JupiterOne Questions library]("https://ask.us.jupiterone.io/filter?tagFilter=all").

dashboard_nuclei_portal.json

@@ -0,0 +1,195 @@
+{
+	"widgets": [
+		{
+			"id": "19a61314-b3b7-4138-bba4-2429aaf5915b",
+			"title": "# of findings",
+			"type": "number",
+			"config": {
+				"queries": [
+					{
+						"name": "query1",
+						"query": "FIND nuclei_finding as f\nRETURN count(f) as value"
+					}
+				]
+			}
+		},
+		{
+			"id": "6110c88c-a763-4fc5-991a-5faf27869259",
+			"title": "Issue graph",
+			"type": "graph",
+			"config": {
+				"queries": [
+					{
+						"name": "query1",
+						"query": "FIND *\nTHAT HAS >> Finding \nTHAT IS >> Vulnerability as vul\nWHERE vul._type = 'nuclei_vulnerability'\nRETURN TREE"
+					}
+				]
+			}
+		},
+		{
+			"id": "df53564c-e179-4703-92e2-f08df4191b2c",
+			"title": "# of affected assets",
+			"type": "number",
+			"config": {
+				"queries": [
+					{
+						"name": "query1",
+						"query": "FIND UNIQUE * as asset\nTHAT HAS >> Finding\nWITH _type = 'nuclei_finding'\nRETURN count(asset) as value"
+					}
+				]
+			}
+		},
+		{
+			"id": "90f1f350-0a0d-4dc1-95c1-9c23fed9d4e4",
+			"title": "Production Critical - Findings",
+			"type": "bar",
+			"config": {
+				"queries": [
+					{
+						"name": "Findings",
+						"query": "FIND * as target\nTHAT HAS >> Finding as f WHERE\ntarget.tag.Production = true AND\ntarget.classification = \"critical\" AND\nf._type = \"nuclei_finding\"\nreturn f.severity as x, count(f) as y"
+					}
+				]
+			}
+		},
+		{
+			"id": "bc7fcab8-bc35-46da-8d16-a07076f4997d",
+			"title": "# of critical assets affected",
+			"type": "number",
+			"config": {
+				"queries": [
+					{
+						"name": "query1",
+						"query": "FIND *\nWITH tag.Production = true AND classification = 'critical' AS asset\nTHAT HAS >> Finding\nWITH _type = 'nuclei_finding'\nRETURN COUNT(asset) as value"
+					}
+				]
+			}
+		},
+		{
+			"id": "6e74fab5-d955-4ea2-bd93-030a46bdc250",
+			"title": "Total Findings",
+			"type": "bar",
+			"config": {
+				"queries": [
+					{
+						"name": "Findings",
+						"query": "FIND * as target\nTHAT HAS >> Finding as f WHERE\nf._type = \"nuclei_finding\"\nreturn f.severity as x, count(f) as y"
+					}
+				]
+			}
+		},
+		{
+			"id": "3e5fb37a-863b-450c-8c2f-87732b62eb3d",
+			"title": "# of findings for critical assets",
+			"type": "number",
+			"config": {
+				"queries": [
+					{
+						"name": "query1",
+						"query": "FIND * as target\nTHAT HAS >> Finding as f WHERE\ntarget.tag.Production = true AND\ntarget.classification = \"critical\" AND\nf._type = \"nuclei_finding\"\nreturn count(f) as value"
+					}
+				]
+			}
+		}
+	],
+	"layouts": {
+		"sm": [],
+		"xs": [],
+		"lg": [
+			{
+				"static": false,
+				"w": 3,
+				"moved": false,
+				"h": 1,
+				"x": 0,
+				"y": 0,
+				"i": "19a61314-b3b7-4138-bba4-2429aaf5915b"
+			},
+			{
+				"static": false,
+				"w": 12,
+				"moved": false,
+				"h": 3,
+				"x": 0,
+				"y": 2,
+				"i": "6110c88c-a763-4fc5-991a-5faf27869259"
+			},
+			{
+				"static": false,
+				"w": 3,
+				"moved": false,
+				"h": 1,
+				"x": 3,
+				"y": 0,
+				"i": "df53564c-e179-4703-92e2-f08df4191b2c"
+			},
+			{
+				"static": false,
+				"w": 4,
+				"moved": false,
+				"h": 1,
+				"x": 7,
+				"y": 1,
+				"i": "90f1f350-0a0d-4dc1-95c1-9c23fed9d4e4"
+			},
+			{
+				"static": false,
+				"w": 3,
+				"moved": false,
+				"h": 1,
+				"x": 3,
+				"y": 1,
+				"i": "bc7fcab8-bc35-46da-8d16-a07076f4997d"
+			},
+			{
+				"static": false,
+				"w": 4,
+				"moved": false,
+				"h": 1,
+				"x": 7,
+				"y": 0,
+				"i": "6e74fab5-d955-4ea2-bd93-030a46bdc250"
+			},
+			{
+				"static": false,
+				"w": 3,
+				"moved": false,
+				"h": 1,
+				"x": 0,
+				"y": 1,
+				"i": "3e5fb37a-863b-450c-8c2f-87732b62eb3d"
+			}
+		],
+		"xl": [],
+		"md": [
+			{
+				"x": 0,
+				"h": 2,
+				"i": "19a61314-b3b7-4138-bba4-2429aaf5915b",
+				"y": 1000000000000,
+				"w": 6
+			},
+			{
+				"x": 0,
+				"h": 2,
+				"i": "6110c88c-a763-4fc5-991a-5faf27869259",
+				"y": 1000000000000,
+				"w": 6
+			},
+			{
+				"x": 0,
+				"h": 2,
+				"i": "df53564c-e179-4703-92e2-f08df4191b2c",
+				"y": 1000000000000,
+				"w": 6
+			},
+			{
+				"x": 0,
+				"h": 2,
+				"i": "90f1f350-0a0d-4dc1-95c1-9c23fed9d4e4",
+				"y": 1000000000000,
+				"w": 6
+			}
+		]
+	}
+}

j1nuclei/__main__.py

@@ -0,0 +1,18 @@
+import logging
+import sys
+import j1nuclei.cli
+
+
+def setup_logger():
+    root = logging.getLogger()
+    root.setLevel(logging.INFO)
+    handler = logging.StreamHandler(sys.stdout)
+    handler.setLevel(logging.INFO)
+    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+    handler.setFormatter(formatter)
+    root.addHandler(handler)
+
+
+if __name__ == '__main__':
+    setup_logger()
+    sys.exit(j1nuclei.cli.main())