diff --git a/.gitignore b/.gitignore index 0e8c263..bd036f0 100644 --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,9 @@ private.pem test-key.jwk public.pem +# Ignore upstream EFF repo +https-everywhere/ + # Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] diff --git a/README.md b/README.md index 10e2a76..1128257 100644 --- a/README.md +++ b/README.md @@ -26,32 +26,15 @@ which will create `test-key.jwk` in your current working directory. 2. Add their domain name and the requested URL to the `onboarded.txt` via PR into this repository. We match the domain based on the landing page of the organization, comparing the `netloc` in a URL with structure `scheme://netloc/path;parameters?query#fragment`. -3. Next, perform a ruleset release as described below. - -### Updating the onion URL for an organization (e.g. if they transition to v3 or rotate URLs) - -1. First update their onion URL in the official SecureDrop directory using the existing process. - -2. Next, perform a ruleset release as described below. - -### Release process - -Generate rulesets via the securedrop.org directory using the `sddir.py` script: +3. Next, generate and sign the update ruleset using the following command (requires signing key, please ping `@emkll` for assistance): ``` -source .venv/bin/activate -python sddir.py +./scripts/generate-and-sign ``` -This populates the `rulesets` directory. Inspect them and check all looks sane. - -To sign the rules, see HTTPS Everywhere docs [here](https://github.com/EFForg/https-everywhere/blob/master/docs/en_US/ruleset-update-channels.md#2-signing-rulesets-with-this-key) for the signing process. In the step where you remove all HTTPS Everywhere rules from `rules` in the git checkout of the `https-everywhere` git repo, you should copy all rules from `rulesets` generated from the above Python script. You do not need to create a trivial rule as described in the HTTPS Everywhere docs. - -For the production rules this signing must be done via the official signing ceremony and the existing SD release key (JWK formatted version of the pubkey is in `release-pubkey.jwk`). There is some internal documentation with more detailed instructions on this, ping `@emkll` if you need to do this. - -Once you have the signature, place the files to serve in the root of the git tree in this repository, and then update the directory listing in `index.html` using the `update_index.sh` shell script in this directory. +4. Commit all files generated by the script above and open a Pull Request to this repository. Once the PR is merged, the rulesets will automatically be deployed to production. -# Verifying +## Verifying changes Inspect the diff. If it looks good, commit the resulting `index.html` and all files to be served. To test locally, run @@ -59,6 +42,6 @@ Inspect the diff. If it looks good, commit the resulting `index.html` and all fi And configure your browser to use `http://localhost:4080/https-everywhere/`. -# Deployment +## Deployment Upon merge the container will be published to `quay.io/freedomofpress` and the new tag will be deployed automatically. diff --git a/scripts/generate-and-sign b/scripts/generate-and-sign new file mode 100755 index 0000000..63409bb --- /dev/null +++ b/scripts/generate-and-sign @@ -0,0 +1,53 @@ +#!/bin/bash +# Utility script to generate the SecureDrop HTTPS Everywhere rulesets, +# used for managing Onion Names for SecureDrop instances. +# +# Much of the business logic is taken verbatim from the EFF HTTPSE repo: +# +# https://github.com/EFForg/https-everywhere/blob/master/docs/en_US/ruleset-update-channels.md#signing +# +set -e +set -u +set -o pipefail + + +# We need the upstream repo by EFF for a few select scripts. +https_everywhere_repo="https-everywhere" +if [[ ! -d "$https_everywhere_repo" ]]; then + echo "Cloning upstream https-everywhere repo for scripts..." + echo "WARNING: Can take a long time! ~10m even on fast connections." + git clone https://github.com/EFForg/https-everywhere +else + echo "Found https-everywhere repo locally, reusing..." +fi + +# Generate the SD rulesets +echo "Generating SecureDrop Onion Name rulesets..." +python3 sddir.py + +# The EFF scripts require paths to be relative, so copy into subdirs. +echo "Copying SecureDrop Onion Name rulesets ..." +rm -f "${https_everywhere_repo}/rules/"*.xml +cp rulesets/*.xml "${https_everywhere_repo}/rules/" +cp public_release.pem "${https_everywhere_repo}/" + +# Switch to upstream subdir, for access to tooling +pushd "$https_everywhere_repo" +sd_rules_dir="securedrop-rules" +rm -rf "$sd_rules_dir" +mkdir "$sd_rules_dir" +python3 utils/merge-rulesets.py +echo "Preparing rulesets for airgapped signature request..." +./utils/sign-rulesets/async-request.sh public_release.pem "$sd_rules_dir" + +# Return to SD ruleset repo root +popd +echo "Copying rules to SecureDrop ruleset repo..." +cp -v "${https_everywhere_repo}/${sd_rules_dir}/"* . + +echo "Updating index for SecureDrop rules..." +./update_index.sh + +echo "Finished. Please review local changes, and commit as appropriate." +# TODO: Not automatically running 'git add *' due to +# https://github.com/freedomofpress/securedrop-https-everywhere-ruleset/issues/20