Posts Tagged ‘boot’

Ubuntu 20.04 and TPM2 encrypted system disk

Thursday, February 18th, 2021

Following the conversation around a past post about Ubuntu 18.04 and TPM2 encrypted system disk (can be found here), I have added a post about using a more modern Ubuntu (20.04) with TPM2. This post includes all the updates (many thanks to the commenter kelderek for setting up a very well working setup!) and has just been tested on a virtual Ubuntu with (virtual) TPM2 device. A note about how to get this setup – sometime in the future.

A very important note, before we begin – This is not enough! To fully prevent your data from being stolen by a bad person with a screwdriver, you will need to add some additional measures, like locking up your BIOS with a password (resetting the BIOS password resets TPM2 data, which is good) ; Signing and using your own embedded GRUB2 (which is not covered here, and maybe one day will be), with your own certificate ; Protect GRUB2 from booth changed using password ; Lock down boot devices in the BIOS (so the evil hacker cannot boot from a USB or CDROM without unlocking the BIOS first) ; Lock the hard drive (or SSD, or NVME) with a security to the BIOS (possible on some models of laptops and hard drives). All these steps are required to complete the security process in order to make the life of a cracker/hacker harder to the point of impossible, when attempting to obtain your data. Without them, the effort is great, but not impossible.
Also note that all the above described actions might lock you out of your own system in case of any hardware failure or forgotten password, so make sure you have a (secure!) backup of your data, or else…

Now – to the process. The idea is this: We add a new key to the cryptsetup – a long one, and this key is stored in TPM2. We add scripts which pull this key out of TPM2 store whenever the system boots. Thanks to some additional comments by Kelderek, we also add some failback, in case of an incorrect key, to allow up to recover and boot using manual key.

I will not wrap all my actions with ‘sudo’ command. All commands need to be called as root, so you can prefix this guide with ‘sudo -s’ or ‘sudo bash’ if you like. I can also assume you ‘cd’ to the root’s directory to protect the root.key file while it is still there. So – ‘sudo’ yourselves, and let’s begin.

Install required packages:

apt install tpm2-tools

Define TPM2 memory space to hold the key:

tpm2_nvdefine -s 64 0x1500016

# This command will define a 64 bit memory space in TPM2, at the above mentioned address

Create a random 64 bit key file:

cat /dev/urandom | tr -dc ‘a-zA-Z0-9’ | head -c 64 > root.key

Save the contents of the key file to TPM2:

tpm2_nvwrite -i root.key 0x1500016

Compare the contents of the TPM and the file, to verify that they are exactly the same:

echo root.key file contents: `cat root.key`
echo The value stored in TPM: `tpm2_nvread 0x1500016`

Identify the encrypted device, and add the key to the LUKS:

lsblk # Identify the owner of 'crypt'. We will assume it is /dev/sda3 in our example
cryptsetup luksAddKey /dev/sda3 root.key

Create a script to pull the key from TPM2:

#!/bin/sh
if [ -f ".tpm2-getkey.tmp" ]; then
# tmp file exists, meaning we tried the TPM this boot, but it didn’t work for the drive and this must be the second
# or later pass for the drive. Either the TPM is failed/missing, or has the wrong key stored in it.
/lib/cryptsetup/askpass "Automatic disk unlock via TPM failed for  () Enter passphrase: "
exit
fi
# No tmp, so it is the first time trying the script. Create a tmp file and try the TPM
touch .tpm2-getkey.tmp
 
tpm2_nvread 0x1500016

This script will be embedded into the future initramfs (or initrd), and will pull the key from TPM2. Now, we need to set its permissions and ownerships:

chown root: /usr/local/sbin/tpm2-getkey
chmod 750 /usr/local/sbin/tpm2-getkey

Create a hook script to initramfs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
cat << EOF > /etc/initramfs-tools/hooks/tpm2-decryptkey
#!/bin/sh
PREREQ=""
prereqs()
{
echo ""
}
case \$1 in
prereqs)
prereqs
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
copy_exec `which tpm2_nvread`
copy_exec /usr/lib/x86_64-linux-gnu/libtss2-tcti-device.so.0.0.0
copy_exec /usr/lib/x86_64-linux-gnu/libtss2-tcti-device.so.0
copy_exec /lib/cryptsetup/askpass
exit 0
EOF

Set file permissions:

chown root: /etc/initramfs-tools/hooks/tpm2-decryptkey
chmod 755 /etc/initramfs-tools/hooks/tpm2-decryptkey

Edit crypttab:

The file /etc/crypttab needs to have an entry added at the end of the line containing the boot volume. For example (taken from the previous post’s comments):

sda3_crypt UUID=d4a5a9a4-a2da-4c2e-a24c-1c1f764a66d2 none luks,discard

should become:

sda3_crypt UUID=d4a5a9a4-a2da-4c2e-a24c-1c1f764a66d2 none luks,discard,keyscript=/usr/local/sbin/tpm2-getkey

First – backup your existing crypttab file:

cp /etc/crypttab /etc/crypttab.backup

If you have a single line in the file (only the boot volume is encrypted), you can use the following command:

sed -i 's%$%,keyscript=/usr/local/sbin/tpm2-getkey%' /etc/crypttab

Otherwise, you will have to append it manually.

Backup the original initrd and create a new one

cp /boot/initrd.img-$(uname -r) /boot/initrd.img-$(uname -r).orig
mkinitramfs -o /boot/initrd.img-$(uname -r) $(uname -r
)

Reboot.

If all works well – you can now delete the file root.key holding a clear text key. You will still have your original manually defined decryption key working, and the initrd will fail-over to handle manual askpass if the automatic TPM2 fails to work, so you should be rather safe there, as well as update-proof.

Notes:

  • The procedure might work on older Ubuntu systems, although it was not tested. A comment in the previous post suggested that the addition of the source
    deb http://cz.archive.ubuntu.com/ubuntu focal main universe
    to your Ubuntu 18.04, the same procedure should work. I did not test it yet.
  • The security of the system is not complete until you handle the other tasks mentioned at the beginning of the text.
  • This procedure does not cover TPM1.2 systems, and will not work without modification on them. I might document such process in the future.

Good luck, and post here if it works for you, or if you have any suggestions!

Recover non-booting Linux

Thursday, July 23rd, 2020

Linux boot sequence includes many stages. Following the BIOS initialisation, the boot loader is being called (GRUB or GRUB-EFI), then GRUB does its magic, and then it loads the kernel and the init RAM FS (initramfs or initrd) to memory. Following that, the kernel is jumped to, and a set of scripts are called from the initramfs, into a ‘pivot-root’ section, where the real OS, running on the real disks is called.

I will give a quick note about troubleshooting initramfs failures. It will fail when an internal script fails and aborts, it will fail with it cannot mount the real OS root filesystem, it may fail by many unique events which I cannot cover here. However – it is important to know that you can “hack” into the process and get a working shell from within.

Most modern Linux systems use ‘dracut’ as the ramdisk generating framework. There are others, but the method I will show works for ‘dracut’ on RHEL/Centos/OEL 6+

on RHEL 6 – adding this to the boot command line would result in a shell:

rdshell rdbreak=[cmdline|pre-udev|pre-trigger|initqueue|pre-mount|mount|pre-pivot|cleanup]

Select one of the break points you want. Each has a meaning, but I will not go into it now. Make sure you remove the GRUB directives for ‘quiet’ and ‘rhgb’ if they exist.

For RHEL7+ you should append the following syntax to the boot command line:

rd.shell rd.break=[cmdline|pre-udev|pre-trigger|initqueue|pre-mount|mount|pre-pivot|cleanup]

You can read more about it in the Fedoraproject Wiki page.

Extracting multi-layered initramfs

Thursday, December 5th, 2019

Modern Kernel specification (can be seen here) defined the initial ramdisk (initrd or initramfs, depends on who you ask) to allow stacking of compressed or uncompressed CPIO archives. It means, in fact, that you can extend your current initramfs by appending a cpio.gz (or cpio) file at the end, containing the additions or changes to the filesystem (be it directories, files, links and anything else you can think about).

An example of this action:

1
2
3
4
rm -Rf /tmp/extrac # Cleanup
mkdir /tmp/extract
cd /tmp/extract
zcat ../test-of-all.cpio.gz | while cpio -id ; do : ; done

This should work (I haven’t tried, and if you do it – make sure you have a copy of the original initramfs file!), and the contents of the directory /tmp/test would be reflected in the initramfs.

This method allows us to quickly modify existing ramdisk, replacing files (the stacked cpio files are extracted by order), and practically – doing allot of neat tricks.

The trickier question, however, is how to extract the stacked CPIO files.
If you create a file containing multiple cpio.gz files, appended, and just try to extract them, only the contents of the first CPIO file would be extracted.

The Kernel can do it, and so are we. The basic concept we need to understand is that GZIP compresses a stream. It means that there is no difference between a file structured of stacked CPIO files, and then compressed altogether, or a file constructed by appending cpio.gz files. The result would be similar, and so is the handling of the file. It also means that we do not need to run a loop of zcat/un-cpio and then again zcat/un-cpio on the file chunk by chunk, but when we decompress the file, we decompress it in whole.

Let’s create an example file:

1
2
3
4
rm -Rf /tmp/extrac # Cleanup
mkdir /tmp/extract
cd /tmp/extract
zcat ../test-of-all.cpio.gz | while cpio -id ; do : ; done

This script will create ten directories called test1 to test10, each containing a single file called test<number>-file. Each of them will both be archived into a dedicated cpio.gz file (named the same) and appended to a larger file called test-of-all.cpio.gz

If we run the following script to extract the contents, we will get only the first CPIO contents:

1
2
3
4
rm -Rf /tmp/extrac # Cleanup
mkdir /tmp/extract
cd /tmp/extract
zcat ../test-of-all.cpio.gz | while cpio -id ; do : ; done

The resulting would be the directory ‘test1’ with a single file in it, but with nothing else. The trick to extract all files would be to run the following command:

1
2
3
4
rm -Rf /tmp/extrac # Cleanup
mkdir /tmp/extract
cd /tmp/extract
zcat ../test-of-all.cpio.gz | while cpio -id ; do : ; done

This will extract all files, until there is no more cpio format remaining. Then the ‘cpio’ command will fail and the loop would end.

Some additional notes:
The ‘:’ is a place holder (does nothing) because ‘while’ loop requires a command. It is a legitimate command in shell.

So – now you can extract even complex CPIO structures, such as can be found in older Foreman “Discovery Image” (very old implementation), Tiny Core Linux (see this forum post, and this wiki note as reference on where this stacking is invoked) and more. This said, for extracting Centos/RHEL7 initramfs, which is structured of uncompressed CPIO appended by a cpio.gz file, a different command is required, and a post about it (works for Ubuntu and RHEL) can be found here.

EDIT: It seems the kernel-integrated CPIO extracting method will not “overwrite” a file with a later layer of cpio.gz contents, so I will have to investigate a different approach to that. FYI.