Replication from unencrypted to encrypted set

This is a POC testing ZFS (unencrypted zvols) from a server to another server (encrypted zvols). Using an old laptop as a target with the encrypted zvols.

On the target I first replicated existing large datasets I already had from a test, to an encrypted zpool to seed the data.

WARNING:

saving the encryption key on the file system is not safe
losing your encryption key means losing your data permanently

create encrypted zvol on target

# zfs create -o encryption=on -o keyformat=passphrase -o keylocation=prompt TANK/ENCRYPTED
Enter passphrase:
Re-enter passphrase:

Seed one snapshot source DATA zvol as a test

using 4.57G only

# zfs send -v TANK/DATA@2020-12-19_06.45.01--2w | zfs recv -x encryption TANK/ENCRYPTED/DATA
full send of TANK/DATA@2020-12-19_06.45.01--2w  estimated size is 4.52G
total estimated size is 4.52G
TIME        SENT   SNAPSHOT     TANK/DATA@2020-12-19_06.45.01--2w
08:39:06   34.4M   TANK/DATA@2020-12-19_06.45.01--2w
08:39:07    115M   TANK/DATA@2020-12-19_06.45.01--2w
08:39:08    279M   TANK/DATA@2020-12-19_06.45.01--2w
...
08:40:49   4.52G   TANK/DATA@2020-12-19_06.45.01--2w
08:40:50   4.54G   TANK/DATA@2020-12-19_06.45.01--2w

# zfs list TANK/ENCRYPTED/DATA
NAME                  USED  AVAIL     REFER  MOUNTPOINT
TANK/ENCRYPTED/DATA  4.59G  1017G     4.57G     /TANK/ENCRYPTED/DATA

# zfs list -t snapshot TANK/ENCRYPTED/DATA
NAME                                          USED  AVAIL     REFER     MOUNTPOINT
TANK/ENCRYPTED/DATA@2020-12-19_06.45.01--2w  17.4M      -     4.57G  -

Seed all snapshots source DATA zvol

ends up using 22G

# zfs destroy TANK/ENCRYPTED/DATA
cannot destroy 'TANK/ENCRYPTED/DATA': filesystem has children
use '-r' to destroy the following datasets:
TANK/ENCRYPTED/DATA@2020-12-19_06.45.01--2w

# zfs destroy -r TANK/ENCRYPTED/DATA

# zfs send -R TANK/DATA@2020-12-19_06.45.01--2w | zfs recv -x encryption TANK/ENCRYPTED/DATA

# zfs list TANK/ENCRYPTED/DATA
NAME                  USED  AVAIL     REFER  MOUNTPOINT
TANK/ENCRYPTED/DATA  22.9G   999G     4.57G  /TANK/ENCRYPTED/DATA

# zfs list -t snapshot TANK/ENCRYPTED/DATA | tail -2
TANK/ENCRYPTED/DATA@2020-12-17_06.45.01--2w  11.2M      -     4.57G  -
TANK/ENCRYPTED/DATA@2020-12-19_06.45.01--2w  11.3M      -     4.57G  -

Create ARCHIVE zvol

# zfs create -o encryption=on -o keyformat=passphrase -o keylocation=prompt TANK/ENCRYPTED/ARCHIVE
Enter passphrase:
Re-enter passphrase:

Seed ARCHIVE/MyDocuments

# zfs send -R TANK/ARCHIVE/MyDocuments@2020-12-18_20.15.01--2w | zfs recv -x encryption TANK/ENCRYPTED/ARCHIVE/MyDocuments

Test sending src zvol from source to target (via ssh)

NOTE: Loading the key manually. Will try automatically later.

on target:
# zfs destroy TANK/ENCRYPTED/ARCHIVE/src@2020-12-19_20.15.01--2w

on source:
# zfs send -i TANK/ARCHIVE/src@2020-12-18_20.15.01--2w TANK/ARCHIVE/src@2020-12-19_20.15.01--2w | ssh rrosso@192.168.1.79 sudo zfs recv -x encryption TANK/ENCRYPTED/ARCHIVE/src
cannot receive incremental stream: inherited key must be loaded

on target:
# zfs load-key -r TANK/ENCRYPTED
Enter passphrase for 'TANK/ENCRYPTED':
Enter passphrase for 'TANK/ENCRYPTED/ARCHIVE':
2 / 2 key(s) successfully loaded

# zfs rollback TANK/ENCRYPTED/ARCHIVE/src@2020-12-18_20.15.01--2w

on source:
# zfs send -i TANK/ARCHIVE/src@2020-12-18_20.15.01--2w TANK/ARCHIVE/src@2020-12-19_20.15.01--2w | ssh rrosso@192.168.1.79 sudo zfs recv -x encryption TANK/ENCRYPTED/ARCHIVE/src

on target:
# zfs list -t snapshot TANK/ENCRYPTED/ARCHIVE/src | tail -2
TANK/ENCRYPTED/ARCHIVE/src@2020-12-18_20.15.01--2w  1.87M      -      238M  -
TANK/ENCRYPTED/ARCHIVE/src@2020-12-19_20.15.01--2w     0B      -      238M  -

Test using key from a file

NOTE: Do this at your own risk. Key loading should probably be done from remote KMS or something safer.

on target:
# ls -l .zfs-key
-rw-r--r-- 1 root root 9 Dec 21 12:49 .zfs-key

on source:
# ssh rrosso@192.168.1.79 sudo zfs load-key -L file:///root/.zfs-key TANK/ENCRYPTED
# ssh rrosso@192.168.1.79 sudo zfs load-key -L file:///root/.zfs-key TANK/ENCRYPTED/ARCHIVE

on target:
# zfs get all TANK/ENCRYPTED | egrep "encryption|keylocation|keyformat|encryptionroot|keystatus"
TANK/ENCRYPTED  encryption            aes-256-gcm            -
TANK/ENCRYPTED  keylocation           prompt                 local
TANK/ENCRYPTED  keyformat             passphrase             -
TANK/ENCRYPTED  encryptionroot        TANK/ENCRYPTED         -
TANK/ENCRYPTED  keystatus             available              -

# zfs get all TANK/ENCRYPTED/ARCHIVE | egrep "encryption|keylocation|keyformat|encryptionroot|keystatus"
TANK/ENCRYPTED/ARCHIVE  encryption            aes-256-gcm              -
TANK/ENCRYPTED/ARCHIVE  keylocation           prompt                   local
TANK/ENCRYPTED/ARCHIVE  keyformat             passphrase               -
TANK/ENCRYPTED/ARCHIVE  encryptionroot        TANK/ENCRYPTED/ARCHIVE   -
TANK/ENCRYPTED/ARCHIVE  keystatus             available                -

** now test with my replication (send/recv) script

ZFS Send To Encrypted Volume