Running Systems

I had one of my computers die a short while ago. I wanted to get the data inside its disk into another computer.

Using the magical and rather cheap USB2SATA I was able to connect the disk, however, the disk was part of a software mirror (md device) and had LVM on it. Gets a little complicated? Not really:

(connect the device to the system)

Now we need to query which device it is:

dmesg

It is quite easy. In my case it was /dev/sdk (don’t ask). It shown something like this:

usb 1-6: new high speed USB device using address 2
Initializing USB Mass Storage driver…
scsi5 : SCSI emulation for USB Mass Storage devices
Vendor: WDC WD80 Model: WD-WMAM92757594 Rev: 1C05
Type: Direct-Access ANSI SCSI revision: 02
SCSI device sdk: 156250000 512-byte hdwr sectors (80000 MB)
sdk: assuming drive cache: write through
SCSI device sdk: 156250000 512-byte hdwr sectors (80000 MB)
sdk: assuming drive cache: write through
sdk: sdk1 sdk2 sdk3
Attached scsi disk sdk at scsi5, channel 0, id 0, lun 0

This is good. The original system was RH4, so the standard structure is /boot on the first partition, swap and then one large md device containing LVM (at least – my standard).

Lets list the partitions, just to be sure:

# fdisk -l /dev/sdk

Disk /dev/sdk: 80.0 GB, 80000000000 bytes
255 heads, 63 sectors/track, 9726 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdk1 * 1 13 104391 fd Linux raid autodetect
/dev/sdk2 14 144 1052257+ 82 Linux swap
/dev/sdk3 145 9726 76967415 fd Linux raid autodetect

Good. As expected. Let’s activate the md device:

# mdadm –assemble /dev/md2 /dev/sdk3
mdadm: /dev/md2 has been started with 1 drive (out of 2).

It’s going well. Now we have the md device active, and we can try to scan for LVM:

# pvscan
…

PV /dev/md2 VG SVNVG lvm2 [73.38 GB / 55.53 GB free]

Activating the VG is a desired action. Notice the name – SVNVG (a note at the bottom):

# vgchange -a y /dev/SVNVG
3 logical volume(s) in volume group “SVNVG” now active

Now we can list the LVs and mount them on our desired location:

]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy%
LogVol00 SVNVG -wi-a- 2.94G
LogVol01 SVNVG -wi-a- 4.91G
VarVol SVNVG -wi-a- 10.00G

Mounting:

mount /dev/SVNVG/VarVol /mnt/

and it’s all ours.

To remove this connected the disk, we need to reverse the above process.

First, we will umount the volume:

umount /mnt

Now we need to disable the Volume Group:

# vgchange -a n /dev/SVNVG
0 logical volume(s) in volume group “SVNVG” now active

0 logical volumes active means we were able to disable the whole VG.

Disable the MD device:

# mdadm –manage -S /dev/md2

Now we can disconnect the physical disk (actually, the USB) and continue with out life.

A note: RedHat systems name their logical volumes using a default name VolGroup00. You cannot have two VGs with the same name! If you activate a VG which originated from RH system and used a default name, and your current system uses the same defaults, you need to connect the disk to an external system (non RH would do fine) and change the VG name using vgrename before you can proceed.

I’m not too happy about it, but we’ve tried to convince him to migrate his data to another server, or at least let us rebuild the server from scratch.

It’s one of those in this link
, holding few virtual machines (VServers), and it appears that the person who built it decided that the system would get a software mirror, and the data would get even better – a stripe! Nice going… 🙂

So, this customer, although we’ve tried putting him on track, lost tons of data (he had no regular backup policy, so it’s around two weeks of 400 hosted sites. Nice… ), and for some reason, he can’t still understand what this person, who has decided that the system volume is more important than the data volume, is a total jerk. He can’t seem to understand why now, after he has a new Raid5 array, he needs this backup. Nice.

So, without saying "told you so", I just keep a hidden smug. It will go away in a day or two.

As part of my voluntary actions, I manage and support Israeli Amateur Radio Committee Internet server. This machine is an old poor machine, custom made about 5-7 years ago, containing Pentium2 300MHz, and 256MB RAM. It serves few hundreds of users, and you can guess for yourself how slow and miserable this machine is.

After
long wait, the committee has decided to purchase a new server. This server, Dell PE 1800, has landed in my own personal lab just two days ago. It’s a nice machine, cheap, considering its abilities, and it’s waiting just to be installed. Or so it was, just up to now.

Mind you that brand PC servers containing more than one CPU can cost around
3K$ and above. This baby has come with a minimal, yet scalable setup, containing only one CPU out of two, 1GB RAM, our of 8GB possible max, and two SCSI HotSwap HDDs, using 2 out of 6 slots. Nice machine. And it was cheap too. Couldn’t complain about it.

At first, I’ve tried using Dell’s CDs. The "Server Setup" CD is supposed to help me prepare the machine to OS installation, either it be Windows, Linux, Novell, etc. I’ve tried using it, preparing it to a new Centos install, when I’ve noticed it didn’t partition quite as I’ve expected. Well, the "Server Setup" tool has decided I would not use Mirrors, and that I would not use LVM, but would use a predefined permanent setup, and that’s all. This machine did not come with a RAID controller, so I’ve had to configure Software RAID. What better time is there than during the install? Dell’s people think otherwise, so I’ve had to boot into a bootable media of Centos 4.1 (my whole installation tree resides on NFS share). The installation was smooth, and worked just like expected. Fast, sleek, smooth. All I’ve ever expected out of Linux installation on a server class PC. Just like it should have been.

I’ve partitioned the system using the following guidelines:

1) Software mirror Raid /dev/md0, containing /boot (150MB)

2) Two stand alone SWAP partitions, 1GB each, one on each HDD. I do not need mirror for the SWAP.

3) Software mirror Raid /dev/md1, containing LVM, expanding all over what’s left of the disk.

4) Logical Volumes "rootvol" (5GB) holding / and "varvol" (6GB) holding /var. Both can be expanded, so I don’t need to worry now about their final sizes.

As said, the installation went great. However, I was not able to boot the system… I just got each time to a hidden maintenance system partition, and it seems my GRUB failed to install itself. Darn.

I’ve booted into rescue mode, and tried to install GRUB manually. Failed. I think (and it’s not the first time I’ve had such problems with GRUB) GRUB is not as good as everyone say it is. It can’t boot into software mirror, and it means it’s not ready for production, as far as I are.

I’ve used YUM to download and install Lilo, and managed easily to convert /etc/lilo.conf.anaconda to
the correct file for Lilo (/etc/lilo.conf), and to run it. Worked great, and the system was able to boot.