Move the top-level volume to a subvolume

• create a snapshot

  1	sudo btrfs subvol snapshot / /@

(“@” seems to be a common name for the main subvolume)

• set the new snapshot as default subvolume

  1 2 3 4

  sudo btrfs subvol list / # outputs: ID 330 gen 1354 top level 5 path @ # use the id from the output above to sudo btrfs subvol set-default 330 /

• if it is not the root volume, skip to the last point of this section

• adapt grub to boot from the new subvolume

  • mount the new subvolume

    1 2 3

    cd /mnt sudo mkdir newroot sudo mount -o subvol=@ /dev/mapper/cryptroot newroot

  • edit newroot/etc/fstab to include subvol=@ in the / options
    This should not be needed theoretically since it is the default volume, but grub-mkconfig did not produce a correct config for me without it.

    1 2	UUID=2240f8f3-3dfa-4aad-a62f-899f5970cb3d / btrfs defaults,subvol=@,noatime,discard,space_cache,autodefrag,compress=lzo 0 0

  • chroot into the new subvolume and update grub

    1 2 3 4 5 6 7

    for i in /dev /dev/pts /proc /sys /run; do sudo mount --bind $i newroot$i; done sudo mount /dev/mapper/cryptboot newroot/boot sudo mount /dev/sdc1 newroot/boot/efi sudo mount --bind /sys/firmware/efi/efivars newroot/sys/firmware/efi/efivars sudo chroot /mnt/newroot grub-mkconfig -o /boot/grub/grub.cfg grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=arch-grub --recheck

  • unmount and reboot (at your own risk of course :D)

• delete the old data from the toplevel

  1 2 3 4 5

  cd /mnt sudo mkdir oldroot sudo mount -o subvolid=5 /dev/mapper/cryptroot oldroot cd oldroot # CAREFULLY delete everything except @ subvolume

Move existing directory to a subvolume

In this example the directory /home/martin/develop will be replaced with a subvolume.

• create the new subvolume

  1 2 3

  sodo mkdir /mnt/toplevel sudo mount -o subvolid=5 /dev/mapper/cryptroot /mnt/toplevel sudo btrfs subvol create /mnt/toplevel/@develop

• move the data to the new subvolume in the most efficient way (as far as I know) by creating reflinks

  1	sudo cp -ax --reflink=always /mnt/toplevel/@/home/martin/develop/. /mnt/toplevel/@develop

• delete the old directory

  1 2	# DO THIS AT YOUR OWN RISK rm -rf /home/martin/develop

• mount the new subvolume instead

  1 2	mkdir /home/martin/develop sudo mount -o subvol=@develop /dev/mapper/cryptroot /home/martin/develop

• add new fstab entry to mount the new subvolume automatically

  1 2	UUID=2240f8f3-3dfa-4aad-a62f-899f5970cb3d /home/martin/develop btrfs defaults,subvol=@develop,noatime,discard,space_cac he,autodefrag,compress=lzo 0 0

Note that the top-level volume always has the ID 5.

Reference: https://btrfs.wiki.kernel.org/index.php/UseCases#Can_I_take_a_snapshot_of_a_directory.3F

Thanks for reading, I hope it was helpful to you :)

Disclaimer

Some things are distribution agnostic, but other things (especially the how to make it boot) are arch linux specific.
Make sure you have a backup of everything before you start this. If you lose data or destroy your system, it is your own fault :D.

I also don’t claim that this setup is fully secure (I only encrypt to protect the data of my customers in case my laptop gets stolen or something, not to protect my super secret secrets from the NSA :) ).

Concept

To unlock the RAID disks with one password, I went for also encrypting the boot partition and store a key to unlock the root filesystem in the initramfs.

I have two SSDs one with 128GB and one with 120GB. The partitions used for raid 0 should be the same size. Different sizes is possible, but the left over size of the bigger partition won’t be used (but we don’t have to make them precisely equal for it to work).

I put boot efi and swap on the larger disk and use two 120GB partitions for the RAID.

Resulting disk layout:

1
2
3
4
5
6
7
8
9
10
11

NAME           FSTYPE         SIZE
sdY                         119,2G
├─sdY1         vfat          1000M
├─sdY2         crypto_LUKS   1000M
│ └─cryptboot  ext4           998M
├─sdY3         swap             4G
└─sdY4         crypto_LUKS  113,3G
  └─cryptroot  btrfs        113,3G
sdX                         111,8G
└─sdX1         crypto_LUKS  111,8G
  └─cryptroot2 btrfs        111,8G

(I’m not going to use the swap … just created it because of the space difference)

Formating the disks

• format the efi partition fat32

• setup LUKS and format the boot partition with ext4

  1 2 3

  sudo cryptsetup luksFormat /dev/sdX2 sudo cryptsetup open /dev/sdX2 cryptboot sudo mkfs.ext4 /dev/mapper/cryptboot

• setup LUKS and format the root partition with btrfs

  1 2 3

  sudo cryptsetup luksFormat /dev/sdX4 sudo cryptsetup open /dev/sdX4 cryptroot sudo mkfs.btrfs /dev/mapper/cryptroot

• setup LUKS on the second drive

  1 2	sudo cryptsetup luksFormat /dev/sdY1 sudo cryptsetup open /dev/sdY1 cryptroot2

• mount first btrfs and add the partition from the other disk to it

  1 2 3 4 5 6 7

  cd /mnt sudo mkdir cryptroot sudo mount -o compress=lzo /dev/mapper/cryptroot cryptroot/ sudo btrfs device add /dev/mapper/cryptroot2 /mnt/cryptroot # doing it like this also works if the drive is added later (the balance can take a while in this case) sudo btrfs balance start -dconvert=raid0 -mconvert=raid1 /mnt/cryptroot

Copy the root filesystem to the new btrfs RAID (or make a new install on it)

Now you can install your system, or copy your existing root to the new raid btrfs.
In this example I will copy an existing root filesystem onto the new btrfs raid.

• create subvolumes (“@” seams to be a common name to use for the main subvolume)

  1 2 3 4 5 6

  sudo btrfs subvol create /mnt/cryptroot/@ sudo btrfs subvol list /mnt/cryptroot/ # output of above: ID 330 gen 1354 top level 5 path @ # use the ID shown to set the "@" subvolume as default sudo btrfs subvol set-default 330 /mnt/cryptroot/

• maybe create more subvolumes (“@home” is common for /home)

• copy the installation

  1 2 3

  sudo mkdir /mnt/newroot sudo mount -o subvol=@ /dev/mapper/cryptroot newroot sudo cp -a /mnt/oldroot/. /mnt/newroot/

(-a .. recursive copy that preserves all the things, like permissions)

• copy kernel to new boot

  1 2 3

  sudo mkdir /mnt/cryptboot sudo mount /dev/mapper/cryptboot /mnt/cryptboot/ sudo cp -a /mnt/oldboot/vmlinuz-linux /mnt/cryptboot/

Make it boot

WARNING: This part is arch linux specific.

Boot will be unlocked by grub via password to get to the initramfs.
The initramfs will unlocked root (it will have the key in it).
Boot must be unlocked again with key in root so that it can be mounted in the “normal” mode (so that an update can store a new kernel and initramfs).

• add a keyfile for unlocking to all LUKS partitions

  1 2 3 4 5 6

  sudo dd bs=512 count=4 if=/dev/random of=/mnt/cryptroot/crypto_keyfile.bin sudo chmod 600 /mnt/cryptboot/cryptokey.bin sudo cryptsetup luksAddKey /dev/sdX2 cryptroot/crypto_keyfile.bin sudo cryptsetup luksAddKey /dev/sdX4 cryptroot/crypto_keyfile.bin sudo cryptsetup luksAddKey /dev/sdY1 cryptroot/crypto_keyfile.bin

WARNING: use this name and path (“/crypto_keyfile.bin”), otherwise additional steps must be taken later (this is the default file name and location).

• chroot into the new system

  1 2 3 4 5

  for i in /dev /dev/pts /proc /sys /run; do sudo mount --bind $i /mnt/newroot$i; done sudo mount /dev/mapper/cryptboot newroot/boot sudo mount /dev/sdY1 newroot/boot/efi sudo mount --bind /sys/firmware/efi/efivars newroot/sys/firmware/efi/efivars sudo chroot /mnt/newroot

• add additional hook that will be used by the initramfs to unlock the second root partition

  1 2 3 4 5 6 7

  # copy the original hook cp /usr/lib/initcpio/install/encrypt /etc/initcpio/install/encrypt2 cp /usr/lib/initcpio/hooks/encrypt /etc/initcpio/hooks/encrypt2 # adapt the new hook to use different names and to NOT delete the keyfile sed -i "s/cryptdevice/cryptdevice2/" /etc/initcpio/hooks/encrypt2 sed -i "s/cryptkey/cryptkey2/" /etc/initcpio/hooks/encrypt2 sed -i "s/rm -f \${ckeyfile}//" /etc/initcpio/hooks/encrypt2

see also: https://wiki.archlinux.org/index.php/Dm-crypt/Specialties#Modifying_the_encrypt_hook_for_multiple_partitions
alternative hook see: https://bbs.archlinux.org/viewtopic.php?id=105086

• add encrypt2 hook to mkinitcpio.conf BEFORE encrypt hook (because encrypt hook deletes the key file at the end)
• edit “/mnt/cryptroot/etc/mkinitcpio.conf” to include your keyfile and add the proper hooks:

  1 2	FILES="/crypto_keyfile.bin" HOOKS=" ... udev encrypt2 encrypt ... " #(before "filesystems")

(encrypt2 before encrypt because encrypt deletes the keyfile from the initial ram disk)

• generate the initramfs

  1 2 3 4

  mkinitcpio -p linux # initramfs includes the key, so only root should be able to read it chmod 600 /boot/initramfs-linux-fallback.img chmod 600 /boot/initramfs-linux.img

• add boot to crypttab (root does not need to be here, it will be unlocked by the intitramfs)

  1	cryptboot UUID=054cc16f-1257-4153-954b-6fea9c1ed931 /crypto_keyfile.bin luks,discard,nofail

• edit /etc/fstab (use UUID of btrfs volume, it is the same for both raid volumes)

  1 2 3

  UUID=8a67292b-24e7-4d4b-ba6e-e4f9f7160524 / btrfs defaults,subvol=@,noatime,discard,compress=lzo,space_cache,autodefrag 0 1 /dev/mapper/cryptboot /boot ext4 defaults,noatime 0 2 UUID=FD95-93A6 /boot/efi vfat defaults 0 0

• edit /etc/defaults/grub add cryptdevice2 with the second device

  1 2	GRUB_CMDLINE_LINUX="cryptdevice=/dev/disk/by-uuid/e506207c-11a0-43f9-9e0b-c212bb6b2fee:cryptroot:allow-discards cryptdevice2=/dev/disk/by-uuid/abc6207c-11a0-43f9-9e0b-c212bb6b2abc:cryptroot2:allow-discards root=/dev/mapper/cryptroot" GRUB_ENABLE_CRYPTODISK=y

• generate grub config and install grub

  1 2	grub-mkconfig -o /boot/grub/grub.cfg grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=arch-grub --recheck

• sync, unmount, reboot and hope for the best :)

Unlock second device with key on first device

In this setup, the first device will be unlocked with password, and the second device with a keyfile inside that first encrypted volume.
A LUKS partition can have up to 8 different keys. So you don’t have to ditch your password for a keyfile, you can have both (or 8 to be more precise ;) ).

• add a key to existing LUKS partition

  1 2 3 4 5 6

  # generate random key with 4kB sudo dd if=/dev/urandom of=/root/keyfile bs=1024 count=4 # set to read only for root sudo chmod 0400 /root/keyfile # add the key to the existing LUKS (prompts for existing passphrase of that LUKS) sudo cryptsetup luksAddKey /dev/sdX /root/keyfile

More about this here: https://www.howtoforge.com/automatically-unlock-luks-encrypted-drives-with-a-keyfile

• add second (none root) drive to crypttab

  1	crypt2 UUID=f2ee83ef-a828-4a84-a150-2ffd781b495a /root/keyfile luks,discard

• check hooks in /etc/mkinitcpio.conf

  1	HOOKS="base udev autodetect modconf block keyboard keymap encrypt lvm2 resume filesystems fsck"

Note: This setup did not work with systemd-hooks (sd-encrypt)

• add the new device to fstab, if it should also be mounted automatically

• rebuild initramfs

  1	sudo mkinitcpio -p linux

• make sure you know how you can get back into your system to fix things if something went wrong and the boot is broken!

• reboot (read on first)

Troubleshooting: System hangs at boot now

If the second unlocked volume contains an LVM, there could be a problem with pvscan, that makes the boot hang.
See this bug report: https://bugs.archlinux.org/task/41833

A workaround for this is to:

• copy udev rule (to override default so that update does not break the workaround)

  1	cp /lib/initcpio/udev/69-dm-lvm-metad.rules /etc/udev/rules.d/

• remove “–background” from pvscan call

  1	RUN+="/usr/bin/lvm pvscan --cache --activate ay --major $major --minor $minor", ENV{LVM_SCANNED}="1"

The above solution is the one that after a lot of trial and error worked for me.
Below are some other things I tried, they required to enter multiple passwords or did not work if the root filesystem is also on encrypted, but maybe it is also helpful to see what does not work.

Unlock with two password promts

• add drives in /etc/crypttab.initramfs (NOTE: not crypttab but crypttab.initramfs, only those are unlocked by the initramfs)

  1 2	cryptAntergos UUID=dfe2737b-fbc3-4aa0-9851-c8eff0c76abd main luks,discard cryptDevelop UUID=f2ee83ef-a828-4a84-a150-2ffd781b495a main luks,discard

• adapt /boot/loader/entries/antergos.conf

  1	options luks.uuid=dfe2737b-fbc3-4aa0-9851-c8eff0c76abd luks.uuid=f2ee83ef-a828-4a84-a150-2ffd781b495a root=/dev/mapper/AntergosVG-AntergosRoot rw quiet

• in /etc/mkinitcpio.conf change the hooks to systemd hooks “encrypt” to “sd-encrypt” and add “systemd” hook before it (also lvm2 -> sd-lvm2 and keymap -> sd-vconsole)

  1	HOOKS="base systemd autodetect modconf block keyboard sd-vconsole sd-encrypt sd-lvm2 filesystems fsck"

• rebuild initramfs

  1	sudo mkinitcpio -p linux

sd-encrypt may gain the ability to cache a password in the future (so that the same password would not have to be entered twice), but does not seam to have this yet (not sure )

Systemd cryptsetup documentation: https://www.freedesktop.org/software/systemd/man/systemd-cryptsetup-generator.html

Unlock two none root LUKS volumes with the same password

• use non-systemd hooks in mkinitcpio.conf
• in crypttab add “keyscript=decrypt_keyctl”

  1 2	cryptAntergos UUID=dfe2737b-fbc3-4aa0-9851-c8eff0c76abd main luks,discard,keyscript=decrypt_keyctl cryptDevelop UUID=f2ee83ef-a828-4a84-a150-2ffd781b495a main luks,discard,keyscript=decrypt_keyctl

See also (for alternative “encrypt” hooks):

• https://bugs.archlinux.org/task/23182
• https://bugs.archlinux.org/task/50901

Thank you for reading, I hope that it was helpful, despite the lacking detailed descriptions :)

This work is licensed under a Creative Commons Attribution 4.0 International License.

Disk layout

All the commands shown later, will be according to this layout.

1
2
3
4
5
6
7
8
9

NAME                  SIZE RO TYPE  Note
sdc                 223,6G  0 disk  Disk that should be used
├─sdc1                256M  0 part  unencrypted fat32 that should be used for /boot/efi
├─sdc2                256M  0 part  unencrypted ext4 that should be used for /boot
└─sdc3              223,1G  0 part  LUKS encrypted partition
  └─develop         223,1G  0 crypt 
    ├─vg01-swap         8G  0 lvm   swap that should be used
    ├─vg01-root       100G  0 lvm   should be used as root /
    └─vg01-develop  115,1G  0 lvm   data partition that should not be touched

Install

• Boot from your install medium in UEFI mode.
• Start the live system (“Try Ubuntu …”)

• Unlock the encrypted partition

  1 2 3 4

  # find partition lsblk # open it sudo cryptsetup luksOpen /dev/sdc3 sdc3_crypt

• run the installer

  • set the root to be /dev/mapper/vg01-root select a filesystem (ext4) and check the format box
  • set /boot as mountpoint for /dev/sdc2
  • set /dev/sdc as boot device
  • install

Manually setup the boot partition

• prepare the new install for chroot

  1 2 3 4 5 6 7 8

  # mount the root partition sudo mount /dev/mapper/vg01-root /mnt sudo mount /dev/sdc2 /mnt/boot sudo mount /dev/sdc1 /mnt/boot/efi # bind the dev, proc, and sys to the new root sudo mount --bind /dev /mnt/dev sudo mount --bind /proc /mnt/proc sudo mount --bind /sys /mnt/sys

• chroot into the new install

  1	sudo chroot /mnt

• find the uuid of the luks encrypted partition

  1	blkid /dev/sdc3

• add a new file /etc/crypttab with the content (where sdc3_crypt is just a name, I used the same here as in the luksOpen above)

  1 2	# <target name><source device><key file><options> sdc3_crypt UUID=f2ee83ef-a828-4a84-a150-2ffd781b495a none luks,discard

The discard option is only relevant for SSD drives and may reduce the security, see the warning in the manpage here.

• fix the boot setup

  1 2 3 4 5 6 7 8 9

  # reinstall grub apt-get install --reinstall grub-efi-amd64 grub-install # update the initramfs update-initramfs -k all -c -v # update grub update-grub

All done

That should be it, reboot the system and there should be an “ubuntu” entry in the UEFI menu, boot from it and it should prompt for the LUKS password and then boot to the normal lightdm login screen.

Thank you for reading, I hope that it was helpful :)

Edit: Post edited on 2016-10-14. Changed from single partition for /boot and /efi to separate partitions and fixed errors.

This work is licensed under a Creative Commons Attribution 4.0 International License.