Linux

Encrypted btrfs RAID as root

This post describes how to setup a fully encrypted root btrfs on two disks, running in RAID 0.

Disclaimer

Some things are distribution agnostic, but other things (especially the how to make it boot) are arch linux specific.
Make sure you have a backup of everything before you start this. If you lose data or destroy your system, it is your own fault :D.

I also don’t claim that this setup is fully secure (I only encrypt to protect the data of my customers in case my laptop gets stolen or something, not to protect my super secret secrets from the NSA :) ).

Concept

To unlock the RAID disks with one password, I went for also encrypting the boot partition and store a key to unlock the root filesystem in the initramfs.

I have two SSDs one with 128GB and one with 120GB. The partitions used for raid 0 should be the same size. Different sizes is possible, but the left over size of the bigger partition won’t be used (but we don’t have to make them precisely equal for it to work).

I put boot efi and swap on the larger disk and use two 120GB partitions for the RAID.

Resulting disk layout:

1
2
3
4
5
6
7
8
9
10
11
NAME           FSTYPE         SIZE
sdY                         119,2G
├─sdY1         vfat          1000M
├─sdY2         crypto_LUKS   1000M
│ └─cryptboot  ext4           998M
├─sdY3         swap             4G
└─sdY4         crypto_LUKS  113,3G
  └─cryptroot  btrfs        113,3G
sdX                         111,8G
└─sdX1         crypto_LUKS  111,8G
  └─cryptroot2 btrfs        111,8G

(I’m not going to use the swap … just created it because of the space difference)

Formating the disks

  • format the efi partition fat32

  • setup LUKS and format the boot partition with ext4

    1
    2
    3
    sudo cryptsetup luksFormat /dev/sdX2
    sudo cryptsetup open /dev/sdX2 cryptboot
    sudo mkfs.ext4 /dev/mapper/cryptboot

  • setup LUKS and format the root partition with btrfs

    1
    2
    3
    sudo cryptsetup luksFormat /dev/sdX4
    sudo cryptsetup open /dev/sdX4 cryptroot
    sudo mkfs.btrfs /dev/mapper/cryptroot

  • setup LUKS on the second drive 

    1
    2
    sudo cryptsetup luksFormat /dev/sdY1
    sudo cryptsetup open /dev/sdY1 cryptroot2

  • mount first btrfs and add the partition from the other disk to it

    1
    2
    3
    4
    5
    6
    7
    cd /mnt
    sudo mkdir cryptroot
    sudo mount -o compress=lzo /dev/mapper/cryptroot cryptroot/

    sudo btrfs device add /dev/mapper/cryptroot2 /mnt/cryptroot
    # doing it like this also works if the drive is added later (the balance can take a while in this case)
    sudo btrfs balance start -dconvert=raid0 -mconvert=raid1 /mnt/cryptroot

Copy the root filesystem to the new btrfs RAID (or make a new install on it)

Now you can install your system, or copy your existing root to the new raid btrfs.
In this example I will copy an existing root filesystem onto the new btrfs raid.

  • create subvolumes (“@” seams to be a common name to use for the main subvolume)

    1
    2
    3
    4
    5
    6
    sudo btrfs subvol create /mnt/cryptroot/@

    sudo btrfs subvol list /mnt/cryptroot/
    # output of above: ID 330 gen 1354 top level 5 path @
    # use the ID shown to set the "@" subvolume as default
    sudo btrfs subvol set-default 330 /mnt/cryptroot/

  • maybe create more subvolumes (“@home” is common for /home)

  • copy the installation
    1
    2
    3
    sudo mkdir /mnt/newroot
    sudo mount -o subvol=@ /dev/mapper/cryptroot newroot 
    sudo cp -a /mnt/oldroot/. /mnt/newroot/

(-a .. recursive copy that preserves all the things, like permissions)

  • copy kernel to new boot
    1
    2
    3
    sudo mkdir /mnt/cryptboot
    sudo mount /dev/mapper/cryptboot /mnt/cryptboot/
    sudo cp -a /mnt/oldboot/vmlinuz-linux /mnt/cryptboot/

Make it boot

WARNING: This part is arch linux specific.

Boot will be unlocked by grub via password to get to the initramfs.
The initramfs will unlocked root (it will have the key in it).
Boot must be unlocked again with key in root so that it can be mounted in the “normal” mode (so that an update can store a new kernel and initramfs).

  • add a keyfile for unlocking to all LUKS partitions 
    1
    2
    3
    4
    5
    6
    sudo dd bs=512 count=4 if=/dev/random of=/mnt/cryptroot/crypto_keyfile.bin
    sudo chmod 600 /mnt/cryptboot/cryptokey.bin

    sudo cryptsetup luksAddKey /dev/sdX2 cryptroot/crypto_keyfile.bin
    sudo cryptsetup luksAddKey /dev/sdX4 cryptroot/crypto_keyfile.bin
    sudo cryptsetup luksAddKey /dev/sdY1 cryptroot/crypto_keyfile.bin

WARNING: use this name and path (“/crypto_keyfile.bin”), otherwise additional steps must be taken later (this is the default file name and location).

  • chroot into the new system

    1
    2
    3
    4
    5
    for i in /dev /dev/pts /proc /sys /run; do sudo mount --bind $i /mnt/newroot$i; done
    sudo mount /dev/mapper/cryptboot newroot/boot
    sudo mount /dev/sdY1 newroot/boot/efi
    sudo mount --bind /sys/firmware/efi/efivars newroot/sys/firmware/efi/efivars 
    sudo chroot /mnt/newroot

  • add additional hook that will be used by the initramfs to unlock the second root partition

    1
    2
    3
    4
    5
    6
    7
    # copy the original hook
    cp /usr/lib/initcpio/install/encrypt /etc/initcpio/install/encrypt2
    cp /usr/lib/initcpio/hooks/encrypt  /etc/initcpio/hooks/encrypt2
    # adapt the new hook to use different names and to NOT delete the keyfile
    sed -i "s/cryptdevice/cryptdevice2/" /etc/initcpio/hooks/encrypt2
    sed -i "s/cryptkey/cryptkey2/" /etc/initcpio/hooks/encrypt2
    sed -i "s/rm -f \${ckeyfile}//" /etc/initcpio/hooks/encrypt2

see also: https://wiki.archlinux.org/index.php/Dm-crypt/Specialties#Modifying_the_encrypt_hook_for_multiple_partitions
alternative hook see: https://bbs.archlinux.org/viewtopic.php?id=105086

  • add encrypt2 hook to mkinitcpio.conf BEFORE encrypt hook (because encrypt hook deletes the key file at the end)
  • edit “/mnt/cryptroot/etc/mkinitcpio.conf” to include your keyfile and add the proper hooks:
    1
    2
    FILES="/crypto_keyfile.bin"
    HOOKS=" ... udev encrypt2 encrypt ... " #(before "filesystems")

(encrypt2 before encrypt because encrypt deletes the keyfile from the initial ram disk)

  • generate the initramfs

    1
    2
    3
    4
    mkinitcpio -p linux
    # initramfs includes the key, so only root should be able to read it
    chmod 600 /boot/initramfs-linux-fallback.img
    chmod 600 /boot/initramfs-linux.img

  • add boot to crypttab (root does not need to be here, it will be unlocked by the intitramfs)

    1
    cryptboot UUID=054cc16f-1257-4153-954b-6fea9c1ed931 /crypto_keyfile.bin luks,discard,nofail

  • edit /etc/fstab (use UUID of btrfs volume, it is the same for both raid volumes)

    1
    2
    3
    UUID=8a67292b-24e7-4d4b-ba6e-e4f9f7160524 / btrfs defaults,subvol=@,noatime,discard,compress=lzo,space_cache,autodefrag 0 1
    /dev/mapper/cryptboot /boot ext4 defaults,noatime 0 2
    UUID=FD95-93A6 /boot/efi vfat defaults 0 0

  • edit /etc/defaults/grub add cryptdevice2 with the second device

    1
    2
    GRUB_CMDLINE_LINUX="cryptdevice=/dev/disk/by-uuid/e506207c-11a0-43f9-9e0b-c212bb6b2fee:cryptroot:allow-discards cryptdevice2=/dev/disk/by-uuid/abc6207c-11a0-43f9-9e0b-c212bb6b2abc:cryptroot2:allow-discards root=/dev/mapper/cryptroot"
    GRUB_ENABLE_CRYPTODISK=y

  • generate grub config and install grub

    1
    2
    grub-mkconfig -o /boot/grub/grub.cfg
    grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=arch-grub --recheck

  • sync, unmount, reboot and hope for the best :)