From 4a849034e27d519cd537211b31c7cca4efbf4020 Mon Sep 17 00:00:00 2001 From: Andrew Sichevoi Date: Sun, 19 Mar 2023 04:09:19 +0100 Subject: [PATCH] release v0.1.4 (#12) --- CHANGES.md | 8 ++++ README.md | 26 +++++++++---- ensure-ansible-vaulted.sh | 25 ++++++++++--- tests/core-smoke.sh | 37 ++++++++++++++++++- .../.ensure-ansiblevaulted.yml | 3 ++ .../payload/core-smoke.test-repo.d/.gitignore | 3 ++ 6 files changed, 89 insertions(+), 13 deletions(-) create mode 100644 tests/payload/core-smoke.test-repo.d/.gitignore diff --git a/CHANGES.md b/CHANGES.md index 22dd67d..ae1404d 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,5 +1,13 @@ # Changes +## v0.1.4 + +- fix: non-working abnormal script termination in a loop (tl;dr exit from a spawned process for a loop rather than from the script itself) + +## v0.1.3 + +- add: `track-git-ignore` (in `.ensure-ansiblevaulted.yml`) to raise a warning/error when a non-encrypted version of file is not listed in `.gitignore` + ## v0.1.2 - fix: error is raised when the config is not available diff --git a/README.md b/README.md index 257236a..b27e023 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # pre-commit hook: ensure encrypted with `ansible-vault` -Unconditional⋆ [`pre-commit`](https://pre-commit.com) hook to encrypt certain files using `ansible-vault` +Unconditional⋆ [`pre-commit`](https://pre-commit.com) hook to encrypt sensitive files using `ansible-vault` -**tl;dr** git hook to encrypt `terraform`'s `tfstate` files on commit using `ansible-vault`. Could be used for other file types as well and as a standalone script. +**tl;dr** git hook to encrypt `terraform`'s `tfstate` files on commit using `ansible-vault`. Could be used for **other file types** as well and as a standalone script. ## Usage @@ -16,7 +16,7 @@ Unconditional⋆ [`pre-commit`](https://pre-commit.com) hook to encrypt certain ``` yaml - repo: https://github.com/thekondor/pre-commit-hook-ensure-ansiblevaulted - rev: v0.1.2 # or other specific tag + rev: v0.1.4 # or other specific tag hooks: - id: ensure-ansiblevaulted ``` @@ -25,8 +25,16 @@ Unconditional⋆ [`pre-commit`](https://pre-commit.com) hook to encrypt certain ``` yaml # Optional. Default: vault-encrypted +# The suffix to be added to an encrypted file created with `ansible-vault` encrypted-suffix: vault +# Optional. Default: .with-error +# Should files to encrypted listed in `files` (see below) be checked against `.gitignore`: +# - `.with-error` fails the hook once a file to process is not added (as a pattern or as a filename) to `.gitignore`. +# Highly recommended to prevent accidental snitching of these files to a git history. +# - `.with-warning` is similar to `.with-error` but raises a warning message instead and continues the hook's flow. +track-git-ignored: .with-warning + # Required. The value is passed "as is" to `ansible-vault`; so it could be an executable script as well. vault-password-file: path/to/ansible-vault-password @@ -38,15 +46,19 @@ files: Once the hook activated, all files across the repo matching either against `*.ext` or `*.*another-ext` to be encrypted using `ansible-vault` to ones `*.ext.vault` or `*.*another-ext.vault` and staged for a commit. -The files before being encrypted, firstly checked for a difference against encrypted copy. Once they differ, they are re-encrypted using `ansible-vault edit`, otherwise skipped. +The files before being encrypted, firstly checked for a difference against an encrypted copy. Once they differ, they are re-encrypted using `ansible-vault edit`, otherwise skipped. -3. **Highly recommended**: add the patterns from the previous step to `.gitignore` to avoid any accidental commit. Files with `encrypted-suffix` should be comitted only. +3. **Highly recommended**: add the patterns from the previous step to `.gitignore` to avoid any accidental commit to git. Files with `encrypted-suffix` are subject for adding to git only. ## Rationale +### Context + While occasionally playing around with a homelab, I faced with a necessity having encrypted `terraform`'s state files before pushing them to a git. Encryption of these files are must since they can (and they do) contain sensitive data. Alternative options would be either using a remote (cloud) backend with encryption support or some other sophisticated one; for homelab that is an overkill, vendor lock or another extra dependency. Using `ansible-vault` already I want to leverage it for this use-case as well to keep everything more or less consistent way. -Hashicorp hasn't invested in a local `tfstate` encryption so far (e.g. [proposal](https://github.com/hashicorp/terraform/issues/9556); there are some more). PR for contribution seems would be a [waste of time](https://github.com/hashicorp/terraform/pull/28603). +Hashicorp hasn't invested in a local `tfstate` encryption so far (e.g. [proposal](https://github.com/hashicorp/terraform/issues/9556); there are some more). PR for contribution seems would [not be worth the efforts](https://github.com/hashicorp/terraform/pull/28603). + +### Alternative Approaches On of the common patterns to deal with use-cases like this one, is to leverage either `git-encrypt` or local git filters. I prefer to have something more simple and more transparent (w/o being worried that the file is pushed unencrypted because of misconfiguration). That's the reason why git's builtin `pre-commit` (as a concept) hook came to rescue. @@ -55,7 +67,7 @@ On of the common patterns to deal with use-cases like this one, is to leverage e - ⋆Unconditional - the hook is always applied independently whether files to "protect" were changed/staged or not; - On fresh repo checkout, the files remain encrypted, so they are subject for manual decryption first; - Files to "protect" (specified in `.ensure-ansiblevaulted.yml`): - - have to be added manually to `.gitignore` to avoid any accidental stage (no corresponding warning/check from the hook yet); + - have to be added manually to `.gitignore` to avoid any accidental stage (see `track-git-ignored`); - are limited with `find` glob-patterns; - Each file is encrypted to a separate file-based artefact (though e.g. they all could be stored to a single `yml`-based vault). diff --git a/ensure-ansible-vaulted.sh b/ensure-ansible-vaulted.sh index 5521764..8000b20 100755 --- a/ensure-ansible-vaulted.sh +++ b/ensure-ansible-vaulted.sh @@ -49,18 +49,33 @@ if [ -n "${OPT_VAULT_PASSWORD_FILE}" ]; then ANSIBLE_VAULT_PASSWORD_ARG="--vault-password-file ${OPT_VAULT_PASSWORD_FILE}" fi +OPT_ENSURE_IGNORED=$(yq '.track-git-ignored // ".with-error"' "${SELF_CFG}") +if [ ! ".with-error" = "${OPT_ENSURE_IGNORED}" ] && [ ! ".with-warning" = "${OPT_ENSURE_IGNORED}" ]; then + echo " - ${_ERR}: unknown value for track-ignored=${OPT_ENSURE_IGNORED}" + exit 1 +fi + export EDITOR="${SELF_EDITOR}" -yq '.files[]' "${SELF_CFG}" | while read -r PLAIN_ENTRY; do - find "${REPO_DIR}" -type f -name "${PLAIN_ENTRY}" | while read -r PLAIN_FULL_PATH; do +while read -r PLAIN_ENTRY; do + while read -r PLAIN_FULL_PATH; do ENCRYPTED_FULL_PATH="${PLAIN_FULL_PATH}.${OPT_ENCRYPTED_SUFFIX}" echo "- ${_INF} ensure ansible vaulted: ${PLAIN_FULL_PATH} -> ${ENCRYPTED_FULL_PATH}" if [ ! -f "${PLAIN_FULL_PATH}" ]; then - echo " + ${_ERR}: ${PLAIN_FULL_PATH} not found, error stop" + echo " + ${_ERR}: ${PLAIN_FULL_PATH} not found/accessible" exit 1 fi + + if ! git check-ignore -q "${PLAIN_FULL_PATH}"; then + echo " + ${PLAIN_FULL_PATH} is not declared in .gitignore" + if [ ".with-error" = "${OPT_ENSURE_IGNORED}" ]; then + echo " + ${_ERR}: this is critical" + exit 1 + fi + fi + if [ ! -f "${ENCRYPTED_FULL_PATH}" ]; then echo " + ${ENCRYPTED_FULL_PATH} not found, creating for you..." # shellcheck disable=SC2086 @@ -87,5 +102,5 @@ yq '.files[]' "${SELF_CFG}" | while read -r PLAIN_ENTRY; do fi echo "ansible-vaulted[to-add]:${ENCRYPTED_FULL_PATH}" - done -done + done < <(find "${REPO_DIR}" -type f -name "${PLAIN_ENTRY}") +done < <(yq '.files[]' "${SELF_CFG}") diff --git a/tests/core-smoke.sh b/tests/core-smoke.sh index 97bd6a2..0a0b99d 100755 --- a/tests/core-smoke.sh +++ b/tests/core-smoke.sh @@ -36,7 +36,7 @@ cp -r "${SELF_DIR}"/payload/core-smoke.test-repo.d/. . cp -r "${SELF_DIR}"/payload/core-smoke.cfg.d/. . ls -la . -### This will also add `.vault-password` which in normal case doesn't belong there +### This will also add `.vault-password` which in normal case doesn't belong to a git repo git add . git commit -m "initial commit" @@ -51,6 +51,7 @@ git diff --name-only --cached | tee "${git_staged_output}" | while read -r stage fi done +echo "--- ❇️ CASE: normal/errorless flow" echo "🔸diff{" echo \ "dirA1/dirA2/repo.another-secret.vault @@ -66,3 +67,37 @@ if [ ! 0 -eq ${DIFF_RC} ]; then else echo "✅ PASSED" fi + +### NOTE: depends on the previous repo status +echo "--- ❇️ CASE: track ignored with warning" +if ! yq e -i '.track-git-ignored = ".with-warning"' .ensure-ansiblevaulted.yml; then + echo "failed: update config" + exit 1 +fi + +touch "foo.not-ignored-secret" + +if ! "${SUT_DIR}"/hook.sh | grep -q "foo.not-ignored-secret is not declared in .gitignore"; then + echo "❌ FAILED" + exit 1 +else + echo "✅ PASSED" +fi + +### NOTE: depends on the previous repo status +echo "--- ❇️ CASE: track ignored with error" +if ! yq e -i '.track-git-ignored = ".with-error"' .ensure-ansiblevaulted.yml; then + echo "failed: update config" + exit 1 +fi + +output="$("${SUT_DIR}"/hook.sh)" +hook_rc=$? +if [ 0 -ne $hook_rc ] && + echo "${output}" | grep -q "foo.not-ignored-secret is not declared in .gitignore" && + echo "${output}" | grep -q "this is critical"; then + echo "✅ PASSED" +else + echo "❌ FAILED" + echo "output: ${output}" +fi diff --git a/tests/payload/core-smoke.cfg.d/.ensure-ansiblevaulted.yml b/tests/payload/core-smoke.cfg.d/.ensure-ansiblevaulted.yml index 8a6f85d..3462b5c 100644 --- a/tests/payload/core-smoke.cfg.d/.ensure-ansiblevaulted.yml +++ b/tests/payload/core-smoke.cfg.d/.ensure-ansiblevaulted.yml @@ -1,7 +1,10 @@ --- encrypted-suffix: vault +track-git-ignored: .with-error vault-password-file: ./.vault-password files: - "*.secret" - "*.another-secret" - repo.new-secret + ### The file is not existent in vanilla repo but to be created during the tests + - "*.not-ignored-secret" diff --git a/tests/payload/core-smoke.test-repo.d/.gitignore b/tests/payload/core-smoke.test-repo.d/.gitignore new file mode 100644 index 0000000..78ac2cb --- /dev/null +++ b/tests/payload/core-smoke.test-repo.d/.gitignore @@ -0,0 +1,3 @@ +*.secret +*.new-secret +*.another-secret