You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jun 23, 2023. It is now read-only.
* Update tooling to use Node.js v10
* Use package-lock instead of shrinkwrap and remove webtask
* Delete config.json
* Add .editorconfig file
* Add a rule to eslint to enforce max line length
* Adjust Babel settings
* Bump up packages and update config
* New webpack config
* Move responses and messages to a config file
* Run scan on reopened PR
* Refactor code to work with AWS Lambda handler
- Add AWS Lambda handler
- Implement consistent module for HTTP responses
- Get Slack URL from ENV variable
- Use async/await statements instead of callbacks
- Remove ES6 module import statements (it was causing issue for a webpack)
- Remove deprecated calls to "new Buffer"
* Add changes to HTML report view
- Only one button to report an issue with a false positive
- Updated messages
- Refreshed view
* Update old tests
- Rename bugs to issues
- Update CLI test
* Add JWT fixtures for tests
* Ignore some of eslint rules for a test file
* Return error with a function
* Regenerate package-lock.json file
* CircleCI - trigger a build before running tests
* Update a readme file to match the v3 release changes
* Resize and compress image in docs
* Adjust tables in a readme file and add TOC
* Ignore .DS_Store files
* Feat: multiple file extensions for a single file type
* docs: docker image support
* docs: AWS Lambda support
* docs: How to support a new file type
* docs: Link to a new documentation from the main readme file
* fix: do not use es6 module syntax yet
* feat: support configuration for obfuscation module
Serverless tool that detects secrets and passwords in your pull requests - one file at a time.
4
-
5
3
[](https://gitter.im/repo-supervisor/Lobby)
The Repo-supervisor is a tool that helps you to detect secrets and passwords in your code. It's as easy to install as adding a new webhook to your Github repository.
10
8
11
-
## Help
9
+
It works in two separate modes. The first one allows us to scan Github pull requests, and the second one works from the command line where it scans local directories.
12
10
13
-
If you need help please visit [Wiki](https://github.com/auth0/repo-supervisor/wiki/) or [Gitter chat](https://gitter.im/repo-supervisor/Lobby).
-[Why doesn't it find any secrets?](#why-doesnt-it-find-any-secrets)
21
+
-[How to add support for new file types?](#how-to-add-support-for-new-file-types)
22
+
-[What is Auth0?](#what-is-auth0)
23
+
-[Create a free account in Auth0](#create-a-free-account-in-auth0)
24
+
-[Issue Reporting](#issue-reporting)
25
+
-[Author](#author)
26
+
-[License](#license)
14
27
15
-
## Command line mode
28
+
## Usage
16
29
17
-
To start using tool without setting up webhooks etc. you can start scanning local directories right after downloading the source code:
30
+
### Pre-requisites
18
31
19
-
```bash
20
-
npm install --no-optional
21
-
npm run build
22
-
npm run cli ./src/
23
-
```
32
+
To start using a tool, download the latest release from the Github releases page. There are two bundles available for both AWS Lambda deployment as well as for the CLI mode. Using CLI mode doesn't require any additional configuration, whereas to use the PR mode, it's necessary to deploy the bundle to AWS Lambda first.
The CLI mode allows scanning local directories with source code to detect secrets and passwords in files. That is the simplest deployment option, and it could become a part of the CI pipeline.
53
40
54
-
## Docker
55
-
56
-
It's possible to run Repo Supervisor inside the Docker container. It gives you more flexibility and you don't need to configure your local environment with Node.JS and npm. At first you need to build up the Docker image:
41
+
Findings might be either returned in the plaintext or JSON format:
GITHUB_TOKEN=<token> JWT_SECRET=<secret> npm run deploy
85
-
```
64
+
### Github Pull Request mode
86
65
87
-
After script was deployed it will return a URL address to your webtask which then you can use to setup a webhook.
66
+
Running a tool in the pull request mode requires to add a new webhook to the Github repository. Webhook should be triggered on a pull request events whenever someone opens, updates, or closes a PR. Therefore, when a scan is triggered, it will update the PR status to either success or failure, depending on findings.
88
67
89
-
_If you want to deploy webtask with profile different than a standard one you should set env. variable called `WT_PROFILE=myprofile` just before or right after `GITHUB_TOKEN` variable._
68
+
Webhook configuration details:
90
69
91
-
## Webhook
70
+
| Setting | Value |
71
+
| ------------ | ------------------ |
72
+
| Payload URL | AWS Lambda URL |
73
+
| Content type |`application/json`|
74
+
| Events type |`Pull requests`|
92
75
93
-
Installing webhook is easy and there is no difference to other webhooks provided by i.e. Zapier or IFTTT.
76
+
Whenever a tool finds security issues, it sets the PR status to error, and it adds a link to view the report. Link to the report is a URL to AWS Lambda deployment with an additional query parameter `?id=<jwt>` that allows to generate the HTML report.
94
77
95
-
Before installing a webhook you need to build and install this tool. As a result `npm run deploy` should return the URL address to your deployed webtask. Point your **Payload URL** to webtask url and you're ready to go.
78
+
Check out a sample report:
96
79
97
-
:exclamation: Please ensure that the Content type for a webhook is set to `application/json`. :exclamation:
Depending on the success or failure of the scan, it will set a proper PR status.
100
83
101
-
> Which events would you like to trigger this webhook?
84
+
**Error - issues detected**
102
85
103
-
-[x] Let me select individual events.
104
-
-[x] Pull request
86
+
<imgsrc="docs/images/pr.ci.status.error.png"/>
105
87
106
-
## Requirements
88
+
**Success - no issues were found**
107
89
108
-
After installing all required packages with `npm` the one additional tool is `wt-cli` to communicate with [webtask.io](https://webtask.io).
90
+
<imgsrc="docs/images/pr.ci.status.success.png"/>
109
91
110
-
If you don't have an account then create a new one, **it's free**. All details related to `wt-cli` are available in the [documentation](https://webtask.io/docs/wt-cli).
It happens sometimes that you can commit secrets or passwords to your repository by accident. The recommended best practice is not commit the secrets, that's obvious. But not always that obvious when you have a big merge waiting to be reviewed.
96
+
## Supported files
121
97
122
-
This tool allows you to setup a `webhook`that waits for the Pull Requests and scans all interesting files to check for leaked secrets. Every time PR is updated it rescans latest changes and generates a report.
98
+
Repo-supervisor aims to decrease the number of false positives as much as possible. It means that it doesn't scan all file types and extensions. Each file is parsed according to its format to extract strings, and this is a context-aware process that requires to use a language tokenizer. The currently supported file types are:
123
99
124
-
Both acknowledge and rejection actions are triggering Slack notification which allows to whether improve or fix secrets detection rules.
125
-
126
-
**Acknowledge** report (YES): Approve that report as valid, detected strings are actual secrets.
127
-
128
-
**Reject** report (NO): Reject report, detected secrets are not credentials but only object identifiers, messages or other not related strings. It will help to improve the false-positives ratio.
100
+
- JSON (.json)
101
+
- JavaScript (.js)
129
102
103
+
We plan to add new file types in the future. Read a documentation on [how to add a new file type](docs/add.new.file.type.md) to learn more.
130
104
131
105
## Security checks
132
106
133
-
Tool is easily extendable by adding new filter and parsers for a specific format, for now we support `json` and `js` parsing.
107
+
This is the list of currently implemented checks in a tool:
134
108
135
-
-**Entropy Meter** - measures the level of entropy for extracted strings. The higher the entropy the higher probability of detecting a secret/password.
3\. Setup webhook URL so it points to `ngrok` URL.
141
+
Verify that the secrets you want to find are inside supported file types. Read more in the [Supported files](#supported-files) section.
167
142
168
-
[Ngrok](https://ngrok.com/) is a really useful tool, it allows you to inspect **every** request send to your ngrok's endpoint so you can verify data in/out.
143
+
### How to add support for new file types?
169
144
145
+
To support a new file type, you need to create a new parser. Some of the file types might require to use external tokenizers because of the complex structure like [JavaScript](src/parser/tokenizer/js/index.js) files. On the other hand, for simple file types, it's pretty straightforward as it was with [JSON](src/parser/tokenizer/json/index.js) files.
170
146
171
-
## Dependencies
172
-
173
-
All required dependencies are enforced in specific versions on the webtask.io platform by using metadata setting.
0 commit comments