Skip to content

Latest commit

 

History

History
79 lines (57 loc) · 4.44 KB

README.md

File metadata and controls

79 lines (57 loc) · 4.44 KB
Raw

Sensitive At-Rest Data Scanner

Scan directories, exports, and backups for sensitive data (like PII and API keys) with Nightfall's data loss prevention (DLP) APIs. Discover what lives at-rest in your data silos.

This service uses Nightfall's data loss prevention (DLP) APIs to scan a folder/directory, backup, or export.

For example, you can scan a backup of your Salesforce instance to detect sensitive data in Salesforce. Salesforce houses high volumes of customer information, support tickets, quotes and files, synced emails, tasks & notes, and much more. This service will (1) send Salesforce backup data to Nightfall to be scanned, (2) run a local webhook server that retrieves sensitive results back from Nightfall, and (3) write the sensitive findings to a CSV file. This output provides a comprehensive report/audit of the sensitive data at-rest in your Salesforce tenant. The same premise extends to any service that allows you to generate a backup or export.

If you'd like a more detailed tutorial or walk-through of how this service works, we recommend reviewing our file scanner tutorial, as the components are largely the same.

Prerequisites

  • Nightfall account - sign up for free if you don't have an account
  • If you are scanning a cloud backup or export, you'll need admin access to the data silos you wish to scan, in order to create a backup or export

Usage

  1. Create a cloud backup/export of the systems you wish to scan. Download the backup and extract it locally. We've compiled instructions for a handful of popular cloud apps below.
  1. Install dependencies. Add the -U flag to ensure you're using the latest versions of these packages.
pip install -r requirements.txt
  1. Create a local ngrok tunnel to point to your webhook server.
./ngrok http 8000
  1. Set your environment variables: your Nightfall API key, your Nightfall signing secret, your Nightfall detection rule UUID, your webhook server URL from ngrok, and the path to your extracted directory/export/backup that you want to crawl.

Your Nightfall detection rule UUID is optional. If not specified, the default rule will detect likely credit card numbers, US social security numbers, and API keys.

export NIGHTFALL_API_KEY=<your_key_here>
export NIGHTFALL_SIGNING_SECRET=<your_secret_here>
export NIGHTFALL_DETECTION_RULE_UUID=<your_uuid_here>
export NIGHTFALL_SERVER_URL=https://<your_subdomain_here>.ngrok.io
export SCAN_DIRECTORY_PATH='/Users/myuser/salesforce-exports/'
  1. Start your webhook server. This runs app.py.
gunicorn app:app
  1. Run your scan.
python scanner.py
  1. Monitor your webhook server output. Once all file scan events have been received and the scan is complete, view your results in results.csv. Each row in the output CSV will correspond to a sensitive finding. Each row will have the following fields, which you can customize in your webhook server in app.py:
  • Upload ID provided by Nightfall
  • An incrementing index
  • Timestamp
  • Filepath
  • Characters before the sensitive finding (for context)
  • Sensitive finding itself
  • Characters after the sensitive finding (for context)
  • Confidence level of the detection
  • Byte range location (character indicies) of the sensitive finding in its parent file
  • Corresponding detection rules that flagged the sensitive finding