Add support for ZFS encryption #467
Conversation
lowjoel
force-pushed
the
add-zfs-support
branch
5 times, most recently
from
59a2544 to
97bf779
Compare
|
Would there be any instructions, how this can be tested? I'd like to aid in this implementation, but am unsure how. Is there something that can be done by third-parties? |
I've packaged my changes for Ubuntu in a PPA. After you install that, it's essentially just using
Testing would be good. But I only have this specific initramfs configuration. |
|
Thank you for the quick reply. I have found the PPA. Unfortunately I have never used Clevis and cannot unpack the sentence
to infer what a test setup would look like. I'm able to set up an LXD/Incus virtual network that contains an Ubuntu VM/system container and a block devices that contains an encrypted ZFS pool plus a separate instance for Clevis. As a tester I would ask myself:
I am having the intuition that Would it seem suitable to add some documentation with this cycle? The information could go to the |
Ah, OK. Have a look at the Arch wiki. In my case, I want to bind the key with both my TPM and with a Yubikey, so I used
If you want a server, I think you're referring to Tang. I haven't set that up; my key unlock is purely held on the TPM and with a Yubikey, there's no network involved in my setup.
Essentially Clevis does most of the heavy lifting. All this PR does is teach Clevis how to unlock ZFS datasets, which is why there isn't significant documentation. The two arguments:
Calling |
|
I'm on it, but give a few months. Thanks for the link and for helping me understand the relationship between Tang and Clevis better. |
lowjoel
force-pushed
the
add-zfs-support
branch
from
97bf779 to
fc9e6a6
Compare
Installing the ZFS integration should not imply installing the LUKS integration.
lowjoel
force-pushed
the
add-zfs-support
branch
from
fc9e6a6 to
4317bd7
Compare
|
this would be a huge help for our organization 👍 |
|
@lowjoel any help you need to wrap this pr up ? |
|
More testing and code review is good. I've been using it for half a year now, but works well for me |
|
@lowjoel As you are requesting more testing I will be playing around with this shortly. Not an Ubuntu user but Debian. My plain is PVE on encrypted ZFS mirror. I have taken a look at your |
|
@deatharse that looks about right! The changes here are written in shell so you probably could get away with just installing your distro clevis, and then just manually install using dpkg |
|
Thanks, I will let you know how I get on as it would be really nice to get this merged.
Not so as
Which isnt a problem as the Debian version hasn't been updated to 21 yet anyway. |
|
@lowjoel So I've had a play. Much like my earlier suggested plan I:
Now I'm not sure if the following is due to PVE or not but upon reboot i recieved:
Okay, so I configured initramfs by adding a file Updated initramfs again and rebooted. Same thing occurred, however after entering the passphrase I saw the network configured output so it seemed like it was processing out-of-order. I decided to install and configure
Configured dropbear, updated initramfs again, and rebooted. Success, unlocked using the tang server. I know its advisable but I do not remember needing the additional steps with LUKS, but was likely just Debian. I will have a further play later by setting up Debian with ZFS on root. I am not an "official reviewer" for the project but one thing i would suggest is in keeping consistency with I hope this is helpful. |
|
@lowjoel Another consistency point @sarroutbi or @sergio-correia would probably pick up on as a nit-pick is the needless use of the |
|
@lowjoel one bug? I have noticed is if you pass the
it seems counter intuitive to me that this flag would cause it to fail in this circumstance. |
|
From further investigation the need for properly configured
which from the file 155 if [ $netcfg_attempted -eq 0 ] && has_tang_pin ${CRYPTTAB_SOURCE}; then
156 netcfg_attempted=1
157 do_configure_networking
158 fiwhere there is obvious LUKS dependencies 108 has_tang_pin() {
109 local dev="$1"
110
111 clevis luks list -d "${dev}" | grep -q tang
112 }Replicating both of these files and modifying the bottom of - clevisloop &
- echo $! >/run/clevis.pid
+ do_configure_networkingAllows it to work as expected after updating initramfs. It might be worthwhile abstracting the networking functions into a |
|
So I've found an issue with unbinding. If you do not use # clevis zfs bind -d rpool -l boot01 tang '{"url": "http://192.168.1.2/"}'
# clevis zfs bind -d rpool -l boot02 tang '{"url": "http://192.168.1.3/"}'You end up with 3 labels in the form: # zfs get all rpool | tail -3
rpool latchset.clevis.label:boot01 [tang server 1 data] local
rpool latchset.clevis:labels boot01 boot02 local
rpool latchset.clevis.label:boot02 [tang server 2 data] local you can then unbind them in the reverse order: # clevis zfs unbind -l boot02 -d rpool
Loading existing key...
Enter existing ZFS password for rpool:
Wiping Clevis data... okunbinds # clevis zfs unbind -l boot01 -d rpool
Loading existing key...
Enter existing ZFS password for rpool:
Wiping Clevis data... okunbinds however if you unbind in sequential order # clevis zfs unbind -l boot01 -d rpool
Loading existing key...
Enter existing ZFS password for rpool:
Wiping Clevis data... okremoves the label and therefor attempting # clevis zfs unbind -l boot02 -d rpool
ERROR: dataset is not bound with Clevis: rpool
Usage: clevis zfs unbind [-f] [-k KEY] -d DATASET [-a] -l LABEL
Unbinds a label from a ZFS dataset:
-f Force unbinding dataset
-d DATASET The ZFS dataset on which to perform unbinding
-a Unbind all labels
-l LABEL The label to unbind
-k KEY Non-interactively read ZFS password from KEY file
-k - Non-interactively read ZFS password from standard inputthe # zfs inherit latchset.clevis.label:boot02 rpoolif having both bound you attempt to unbind all: # clevis zfs unbind -a -d rpoolremoves the label for |
|
Thanks for this @deatharse -- you've given lots of good observations. Let me find a weekend to address them. |
|
I've had a bit of a further play by creating a second encrypted pool and noticed that was not unlocked. I notice The diff for 22c22,41
< clevis zfs unlock -d "${ENCRYPTIONROOT}"
---
> zpool import -a
>
> for pool in $(zpool list -H -o name); do
> # if pool encryption is active and the zfs command understands '-o encryption'
> if [ "$(zpool list -H -o feature@encryption ${pool})" = 'active' ]; then
> # if the root dataset has encryption enabled
> ENCRYPTIONROOT=$(zfs get -H -o value encryptionroot "${pool}")
> if ! [ "${ENCRYPTIONROOT}" = "-" ]; then
> KEYSTATUS="$(zfs get -H -o value keystatus "${ENCRYPTIONROOT}")"
> # continue only if the key needs to be loaded
> [ "$KEYSTATUS" = "unavailable" ] || continue
> # decrypt them
> TRY_COUNT=5
> while [ $TRY_COUNT -gt 0 ]; do
> clevis zfs unlock -d "${ENCRYPTIONROOT}" && break
> TRY_COUNT=$((TRY_COUNT - 1))
> done
> fi
> fi
> doneN.B. I have yet to create an unencrypted pool with an encrypted dataset, maybe I will get round to that on a weekend. |
|
Heres an updated diff for 22c22,58
< clevis zfs unlock -d "${ENCRYPTIONROOT}"
---
> attempt_unlock() {
> local dataset=$1
>
> KEYSTATUS=$(zfs get -H -o value keystatus "${dataset}")
> # continue only if the key needs to be loaded
> [ "$KEYSTATUS" = "unavailable" ] || break
> # decrypt them
> TRY_COUNT=5
> while [ $TRY_COUNT -gt 0 ]; do
> clevis zfs unlock -d "${dataset}" && break
> TRY_COUNT=$((TRY_COUNT - 1))
> done
> }
>
> zpool import -a
> for pool in $(zpool list -H -o name); do
> # if pool encryption is active and the zfs command understands '-o encryption'
> if [ $(zpool list -H -o feature@encryption "${pool}") = 'active' ]; then
> # if the root dataset has encryption enabled
> ENCRYPTIONROOT=$(zfs get -H -o value encryptionroot "${pool}")
> if ! [ "${ENCRYPTIONROOT}" = "-" ]; then
> attempt_unlock "${ENCRYPTIONROOT}"
> else
> # encryption in child dataset, lets get list of datasets in pool
> for dataset in $( zfs list -r -H -o name "${pool}" ); do
> # first entry will not be a child so we will ignore it
> if [ "${dataset}" != "${pool}" ]; then
> # test for encrypted dataset
> ENCRYPTIONROOT=$(zfs get -H -o value encryptionroot "${dataset}")
> if ! [ "${ENCRYPTIONROOT}" = "-" ]; then
> attempt_unlock "${ENCRYPTIONROOT}"
> fi
> fi
> done
> fi
> fi
> doneCreated test datasets via: # zpool create \
-o ashift=12 \
-o autotrim=on \
-O acltype=posixacl -O xattr=sa -O dnodesize=auto \
-O compression=zstd \
-O normalization=formD \
-O relatime=on \
-O canmount=off \
-O recordsize=1M storage-pool2 /dev/vdd
# zfs create storage-pool2/unencrypted
# zfs create -o encryption=on -o keylocation=prompt -o keyformat=passphrase storage-pool2/encrypted
# zfs create storage-pool2/encrypted/backup
# clevis zfs bind -d storage-pool2/encrypted -l boot02 tang '{"url": "http://192.168.1.3/"}' |
|
Just curious here why the initramfs-tools integration has not added a script into The zfs-initramfs package code executed in initramfs is here. |
|
The recently linked issue has a comment #462 (comment) that reads:
Would that observed behaviour of this branch be intentional, misunderstood or an error? |
Yep, that is my current workaround: #!/bin/sh
# /etc/zfs/initramfs-tools-load-key.d/clevis
# Unlock a ZFS encryption root using Clevis-wrapped key material.
#
# How to provision the clevis properties on a dataset:
# Example (TPM+SB):
# 1. Generate a new wrapping for your dataset key:
# - For passphrase keyformat:
# echo -n 'your-passphrase' | clevis encrypt tpm2 '{"pcr_bank":"sha256","pcr_ids":"7"}' > passphrase.jwe
# zfs set clevis:passphrase="$(cat passphrase.jwe)" pool/dataset
#
# - For hex keyformat:
# echo -n 'your-hex-key' | clevis encrypt {tpm2,tang,sss} '{/*config*/}' > hex.jwe
# zfs set clevis:hex="$(cat hex.jwe)" pool/dataset
#
# - For raw keyformat:
# cat your-raw.key | clevis encrypt {tpm2,tang,sss} '{/*config*/}' > raw.jwe
# zfs set clevis:raw="$(cat raw.jwe)" pool/dataset
#
# 2. Verify:
# zfs get clevis:passphrase pool/dataset
# zfs get clevis:hex pool/dataset
# zfs get clevis:raw pool/dataset
#
# 3. Ensure the dataset's keyformat matches the property you set.
# Example: zfs get keyformat pool/dataset
#
# Environment (from zfs-initramfs):
# ENCRYPTIONROOT : encryption root dataset
# ZFS : helper to run the zfs binary
#
# Contract:
# Return 0 if we did nothing or successfully unlocked; 1 if we tried and failed.
# Helper: read a property; empty if '-' or error.
get_prop() {
val="$($ZFS get -H -o value "$1" "$ENCRYPTIONROOT" 2>/dev/null || true)"
[ "$val" = "-" ] && val=""
printf '%s' "$val"
}
# passphrase | hex | raw | none
KEYFORMAT="$(get_prop keyformat)"
# Build clevis property name directly from keyformat
CLEVIS_PROP="clevis:$KEYFORMAT"
# Fetch the JWE from that property; log whether found or not.
PROP="$(get_prop "$CLEVIS_PROP")"
if [ -n "$PROP" ]; then
log_success_msg "ZFS: found $CLEVIS_PROP on $ENCRYPTIONROOT"
else
log_warning_msg "ZFS: no $CLEVIS_PROP found on $ENCRYPTIONROOT"
return 0
fi
# Optional: log the Clevis pin type (e.g., tpm2/tang/sss) if jose is present
if command -v jose >/dev/null 2>&1; then
PIN="$(printf %s "$PROP" \
| jose jwe fmt -i- \
| jose fmt -j- -Og protected -yOg clevis -Og pin -Su- 2>/dev/null || true)"
if [ -n "$PIN" ]; then
log_success_msg "ZFS: clevis pin=$PIN keyformat=$KEYFORMAT for $ENCRYPTIONROOT"
fi
fi
# Need clevis to decrypt
if ! command -v clevis >/dev/null 2>&1; then
log_warning_msg "ZFS: clevis not available in initramfs; skipping"
return 0
fi
# Unlock using clevis → zfs (zfs interprets bytes according to keyformat)
log_begin_msg "ZFS: unlocking $ENCRYPTIONROOT with clevis ($CLEVIS_PROP)"
printf %s "$PROP" \
| clevis decrypt \
| $ZFS load-key -L prompt "$ENCRYPTIONROOT"
ret=$?
log_end_msg $ret
return $ret |
8 participants
Supersedes #373. Description copied:
Further work by @lowjoel:
clevis-zfs-bind,clevis-zfs-unlock(at reboot), andclevis-zfs-unbindOnce we're happy with the code I can squash the commits to those by @techhazard and myself.