Posts Tagged ‘boot image’

Extracting multi-layered initramfs

Thursday, December 5th, 2019

Modern Kernel specification (can be seen here) defined the initial ramdisk (initrd or initramfs, depends on who you ask) to allow stacking of compressed or uncompressed CPIO archives. It means, in fact, that you can extend your current initramfs by appending a cpio.gz (or cpio) file at the end, containing the additions or changes to the filesystem (be it directories, files, links and anything else you can think about).

An example of this action:

1
2
3
4
5
mkdir /tmp/test
cd /tmp/test
tar -C /home/ezaton/test123 -cf - . | tar xf - # Clones the contests of /home/ezaton/test123 to this location
find ./ | cpio -o -H newc > ../test.cpio.gz # Creates a compressed CPIO file
cat ../test.cpio.gz >> /boot/initramfs-`uname -r`.img

This should work (I haven’t tried, and if you do it – make sure you have a copy of the original initramfs file!), and the contents of the directory /tmp/test would be reflected in the initramfs.

This method allows us to quickly modify existing ramdisk, replacing files (the stacked cpio files are extracted by order), and practically – doing allot of neat tricks.

The trickier question, however, is how to extract the stacked CPIO files.
If you create a file containing multiple cpio.gz files, appended, and just try to extract them, only the contents of the first CPIO file would be extracted.

The Kernel can do it, and so are we. The basic concept we need to understand is that GZIP compresses a stream. It means that there is no difference between a file structured of stacked CPIO files, and then compressed altogether, or a file constructed by appending cpio.gz files. The result would be similar, and so is the handling of the file. It also means that we do not need to run a loop of zcat/un-cpio and then again zcat/un-cpio on the file chunk by chunk, but when we decompress the file, we decompress it in whole.

Let’s create an example file:

1
2
3
4
5
6
cd /tmp for i in {1..10} ; do
    mkdir test${i}
    touch test${i}/test${i}-file
    find ./test${i} | cpio -o -H newc | gzip > test${i}.cpio.gz
    cat test${i}.cpio.gz >> test-of-all.cpio.gz 
done

This script will create ten directories called test1 to test10, each containing a single file called test<number>-file. Each of them will both be archived into a dedicated cpio.gz file (named the same) and appended to a larger file called test-of-all.cpio.gz

If we run the following script to extract the contents, we will get only the first CPIO contents:

1
2
3
mkdir /tmp/extract
cd /tmp/extract
zcat ../test-of-all.cpio.gz | cpio -id # Format is newc, but it is auto detected

The resulting would be the directory ‘test1’ with a single file in it, but with nothing else. The trick to extract all files would be to run the following command:

1
2
3
4
rm -Rf /tmp/extrac # Cleanup
mkdir /tmp/extract
cd /tmp/extract
zcat ../test-of-all.cpio.gz | while cpio -id ; do : ; done

This will extract all files, until there is no more cpio format remaining. Then the ‘cpio’ command will fail and the loop would end.

Some additional notes:
The ‘:’ is a place holder (does nothing) because ‘while’ loop requires a command. It is a legitimate command in shell.

So – now you can extract even complex CPIO structures, such as can be found in older Foreman “Discovery Image” (very old implementation), Tiny Core Linux (see this forum post, and this wiki note as reference on where this stacking is invoked) and more. This said, for extracting Centos/RHEL7 initramfs, which is structured of uncompressed CPIO appended by a cpio.gz file, a different command is required, and a post about it (works for Ubuntu and RHEL) can be found here.

DSL (Damn Small Linux) Diskless boot

Friday, August 31st, 2007

I have come across a requirement to boot a thin client on a very cheap hardware into Linux. Due to the tight hardware requirements, and the tight budget, I have decided to focus on diskless systems, which can be easily modified and purchased to our needs.

Not only that, but due to the hardware configuration (Via 333MHz, 128MB RAM, etc) I have decided to focus on a miniature Linux system.

I dislike re-doing what someone else has done, unless I can do it noticeably better. I have decided to use DSL (Damn Small Linux) as my system of choice, with only minor changes to fit my needs:

Out of the “box”, I was unable to find network-boot DSL. Quickly searching their site, the version which seemed to fit was the initrd-only system. I downloaded it from this mirror, but you can find it as the dsl-x.x.x-initrd.iso file.

Extracting the initrd from the ISO file is quite simple:

mkdir /mnt/iso
mount -o loop dsl-x.x.x-initrd.iso /mnt/iso

And from here you can just copy the contents of the directory /mnt/iso/boot/isolinux/ selectively to your tftpboot directory.

So I got 50MB initrd which worked just fine. Changing this was quite a procedure, because in addition to the steps per the wiki hacking guide, I was required to extract the KNOPPIX file outside of the initrd, and repackage it when done. Quite messy, however, stand-alone as soon as the system has been able to boot.

An alternate I have decided to investigate into was of booting into nfs mount, aka, accessing the KNOPPIX iso disk through NFS and not through CDROM.

I was able to find some leads in DSL forums at this page, which lead to this guide. I was able to download pxe boot image from Knoppix themselves, however, it was based on an old kernel (2.4.20-XFS) which was part of Knoppix 3.3 (cannot find it anymore) and although reached the level of actually booting my nfs, didn’t include enough network drivers (I wanted pcnet32 to be able to “play” with VMware for the task), and was incompatible with my existing DSL.

I had opened the supplied Knoppix initrd, and replaced the modules version to the one supplied with DSL – 2.4.24, per the rest of the system. In addition, I have added my required modules, etc, and was able to boot successfully both on VMware and on the thin client hardware.

To replace the modules, one needs to follow these general-only guidelines (these are not exactly step-by-step instructions):

Mount through loop the DSL KNOPPIX image, for example, in /mnt/dsl
Uncompress the Knoppix PXE initrd
Mount through loop the uncompressed Knoppix PXE initrd, for example, in /mnt/initrd
cd to /mnt/initrd/modules
Replace all modules in the current tree with the ones supplied by DSL, obtainable from /mnt/dsl/lib/modules/2.4.26 directory tree, including the cloop.o module
Umount the initrd image
Compress the initrd image
Boot using DSL linux and the new initrd image.

In order to boot successfully, you need to supply the pxe boot these two instructions:

nfsdir=nfs-server:/path/to/KNOPPIX directory

(since I was quite unsure about the letter case required, I have created a symlink from lower-case to upper case, so I had a link /mnt/KNOPPIX to a directory /mnt/knoppix, and inside this directory, a file called knoppix and a symbolic link to this file KNOPPIX. In my case, the exported path was /mnt/ only. Notice this one!).

BOOT_IMAGE=KNOPPIX – but you can have different KNOPPIX images for different purposes.

Finally it has worked correctly. Changes can be done only to the KNOPPIX iso image, per the hacking guide.

This is my PXE-enabled initrd, based on the text above, which fits DSL-3.4.1: minirt24.gz