Skip to content

Commit 7b6784e

Browse files
AndreiDreyerAndreiDreyer
authored
Merge pull request #110 from joernio/andrei/frontend-docs
Initial Frontend Arg Documentation
2 parents 58f15b9 + ea062c6 commit 7b6784e

File tree

5 files changed

+114
-0
lines changed

5 files changed

+114
-0
lines changed

‎docs.joern.io/content/frontends/_index.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
id: frontends
3+
title: Frontends
4+
bookCollapseSection: true
5+
weight: 135
6+
---
7+
8+
## Frontends
9+
10+
Joern contains multiple frontends, each responsible for parsing the source code and generating the AST for their respective language. `X2CPG` is the base on which all of the other frontends are built, and contains common structures by all of the languages. Each frontend contains the base arguments from `x2cpg`, as well as custom arguments that are frontend specific. See each respective languages' page for a detailed explanation on what arguments are available and what each of the arguments do.
11+
12+
## Invoking a specific frontend
13+
To generate the AST for for some program with a specific frontend, invoke the desired frontend from the `joern-cli/frontends` directory using the generated script. If the script is not there, make sure that Joern has been staged with `sbt`. As an example, to parse a given Python directory and generate an AST:
14+
```bash
15+
cd joern-cli/frontends/pysrc2cpg/
16+
sbt scala stage
17+
./pysrc2cpg /some/input/path --output someOutput --venvDir /some/venv/dir
18+
```
19+
The arguments given after the `pysrc2cpg` command can be a mixture of the args found in `x2cpg`, as well as the language specific args for Python.
20+
21+
If you would like to generate the full `CPG` for a given source directory, you can do so using the `joern-parse` command:
22+
```bash
23+
./joern-parse /some/input/path --language PYTHONSRC --frontend-args --venvDir /some/venv/dir
24+
```
25+
The language argument selects the frontend used to generate the AST. If this argument is omitted, `joern-parse` will select a frontend based on the supported file type with the largest number of files in the given input directory. The arguments passed after `--frontend-args` will be passed to the frontend that is being invoked.
26+
27+
Below is a list of each of the different frontends and the languages that they support:
28+
| **Frontend** | **Language** | **Language Arg** |
29+
| - | - | - |
30+
| `c2cpg` | `C` | `--language C` |
31+
| `csharp2cpg` | `C#` | `--language CSHARPSRC` |
32+
| `ghidra2cpg` | `Ghidra` | `--language GHIDRA` |
33+
| `gosrc2cpg` | `Golang` | `--language GOLANG` |
34+
| `javasrc2cpg` | `Java` | `--language JAVASRC` |
35+
| `jimple2cpg` | `Java` | `--language JAVA` |
36+
| `jssrc2cpg` | `Javascript` | `--language JAVASCRIPT` |
37+
| `kotlin2cpg` | `Kotlin` | `--language KOTLIN` |
38+
| `php2cpg` | `PHP` | `--language PHP` |
39+
| `pysrc2cpg` | `Python` | `--language PYTHONSRC` |
40+
| `rubysrc2cpg` | `Ruby` | `--language RUBYSRC` |
41+
| `swiftsrc2cpg` | `Swift` | `--language SWIFTSRC` |

‎docs.joern.io/content/frontends/java.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
---
2+
id: java
3+
title: Java
4+
weight: 10
5+
---
6+
7+
### Frontend CLI Args
8+
The following arguments are specific to the `javasrc2cpg` frontend.
9+
10+
| **Arg** | **Description** | **Type** | **Example** | **Hidden** |
11+
| - | - | - | - | - |
12+
| `inference-jar-paths` | Paths to extra jars used only for type information | `List<String>` | `--inference-jar-paths "/path/ex1.jar, /path/ex2.jar"` | `false` |
13+
| `fetch-dependences` | Attempts to fetch dependencies jars for extra type information | - | `--fetch-dependencies` | `false` |
14+
| `delombok-java-home` | Optional override to set java home used to run Delombok. Recommended to use Java 17 for best results| `String` | `--delombok-java-home /path/to/home` | `false` |
15+
| `delombok-mode` | Specifies how Delombok should be executed | `no-delombok \| default \| types-only \| run-delombok` | `--delombok-mode no-delombok` | `false` |
16+
| `enable-type-recovery` | Enable generic type recovery. | - | `--enable-type-recovery` | `true` |
17+
| `jdk-path` | JDK path used for resolving builtin Java types. If not set, current classpath will be used | `String` | `--jdk-path /path/to/jdk` | `false` |
18+
| `show-env` | Print information about the environment variables used by `javasrc2cpg` and exit Joern | - | `--show-env` | `false` |
19+
| `skip-type-inf-pass` | Skip the type inference pass. Results will be much worse, so should only be used for development. | - | `--skip-type-inf-pass` | `true` |
20+
| `dump-javaparser-asts` | Dump the `javaparser` ASTs for the given input files and exit Joern. Used for debugging | - | `--dump-javaparser-asts` | `true` |
21+
| `cache-jdk-type-solver` | Re-use the JDK type solver between scans | - | `--cache-jdk-type-solver` | `true` |
22+
| `keep-type-arguments` | Type full names of variables keep their type arguments | - | `--keep-type-arguments` | `true` |
23+
| `no-dummyTypes` | Disables the generation of dummy types during type propagation | - | `--no-dummyTypes` | `true` |
24+
| `type-prop-iterations` | Maximum iterations of type propagation | `Integer` | `--type-prop-iterations 2` | `true` |

‎docs.joern.io/content/frontends/javascript.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
id: javascript
3+
title: JavaScript
4+
weight: 20
5+
---
6+
7+
### Frontend Args
8+
The following arguments are specific to the `jssrc2cpg` frontend.
9+
10+
| **Arg** | **Description** | **Type** | **Example** | **Hidden** |
11+
| - | - | - | - | - |
12+
| `no-tsTypes` | Disable generation of types via `TypeScript` | - | `--no-tsTypes` | `true` |
13+
| `no-dummyTypes` | Disables the generation of dummy types during type propagation | - | `--no-dummyTypes` | `true` |
14+
| `type-prop-iterations` | Maximum iterations of type propagation | `Integer` | `--type-prop-iterations 2` | `true` |

‎docs.joern.io/content/frontends/python.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
id: python
3+
title: Python
4+
weight: 30
5+
---
6+
7+
### Frontend CLI Args
8+
The following arguments are specific to the `pysrc2cpg` frontend.
9+
10+
| **Arg** | **Description** | **Type** | **Example** | **Hidden** |
11+
| - | - | - | - | - |
12+
| `venvDir` | Virtual environment directory. If not absolute it is interpreted relative to `input-dir` | `String` | `--venvDir "/some/path/venv/"` | `true` |
13+
| `venvDirs` | Virtual environment directories. If not absolute it is interpreted relative to `input-dir` | `List<String>` | `--venvDirs "/some/path/venv1/, /some/path/venv2"` | `false` |
14+
| `ignoreVenvDir` | Specifies whether `venv-dir` is ignored. Default to `true` | - | `--ignoreVenvDir` | `false` |
15+
| `ignore-paths` | Ignores specified paths from analysis. If not absolute it is interpreted relative to `input-dir` | `List<String>` | `--ignore-paths "/path/to/ignore1, /path/to/ignore2"` | `false` |
16+
| `ignore-dir-names` | Exclude all files where the relative path from `input-dir` contains at least one of the names specified | `List<String>` | `--ignore-dir-names "rel/path/ignore1, rel/path/ignore2"` | `true` |
17+
| `no-dummyTypes` | Disables the generation of dummy types during type propagation | - | `--no-dummyTypes` | `true` |
18+
| `type-prop-iterations` | Maximum iterations of type propagation | `Integer` | `--type-prop-iterations 2` | `true` |

‎docs.joern.io/content/frontends/x2cpg.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
id: x2cpg
3+
title: X2CPG
4+
weight: 5
5+
---
6+
7+
## Frontend Args
8+
The frontend arguments for `x2cpg` is shared across all frontends for `Joern`. See the individual Frontend pages for frontend-specific args.
9+
10+
| **Arg** | **Description** | **Type** | **Example** |
11+
| - | - | - | - |
12+
| `input-dir` | Directory containing all source files to be used with Joern | `String` | `"/some/path/to/folder"` |
13+
| `output` | Name of the file where output is written to | `String` | `--output "output.txt"` |
14+
| `exclude` | List of input file names to be excluded during CPG generation (paths must be relative to `input-dir` or absolute paths) | `List<String>` | `--exclude "test1.java, test2.java, ..."` |
15+
| `exclude-regex` | A regex specifying files to be excluded during CPG generation (paths must be relative to `input-dir` or absolute paths) | `String` | `--exclude-regex ".*-test.java"` |
16+
| `enable-early-schema-checking` | Enables early schema validation during AST creation (disabled by default)| - | `--enable-early-schema-checking` |
17+
| `enable-file-content` | Add the raw source code to the content of the FILE nodes to allow for method source retrieval via offset fields (disabled by default) | - | `--enable-file-content` |

0 commit comments

Comments
 (0)