Running Systems

I’m in the process of installing VCS on two guest (virtual) Linux nodes, each using the following setup:

256MB RAM

One local (Virtual) HDD, SCSI channel 0

Two shared (Virtual) HDDs, SCSI channel 1

Two NICs, one bridged, and one using /dev/vmnet2, a personal Virtual switch.

The host (carrier) is a Pentium 3 800MHz, with 630MB RAM. I don’t expect great performance out of it, but I expect a slow working testing environment.

Common mistake – Never forget to change your /etc/redhat-release to state you are running a RedHat Enterprise system. Failure to do so will result in failure to install VRTSllt, which will force you to either install it manually after you’ve fixed the file, or remove and reinstall all. In my case, Centos 3.6 (equivalent to RedHat Enterprise Server 3 Update 6) the file /etc/redhat-release should have contained the string:

Red Hat Enterprise Linux AS release 3 (Taroon)

Veritas has advanced a great deal in the last few years, regarding ease of installation, even on multiple servers. Installing Cluster software usually involves more than one server, and installing it all, from a single point, is an advancement. My two nodes are named vcs01-rhel and vcs02-rhel, and the installation was done completely from vcs01-rhel.

The installer assumes you can login using ssh (by default) without password prompt from one node to the other. In my case, it wasn’t true. I’ve found it quicker (and dirty, mind you!) to allow, for the sake of the installation and configuration process, to utilize rsh. It’s not safe, it’s not good, but if it’s just for the short and limited time required for the installation, I’de hack it so it would work. How did I do it?

One node vcs02-rhel I’ve installed (using yum, of course) the package rsh-server. The syntax is yum install rsh-server. Afterwards, I’ve changed its relevant xinetd file, /etc/xinetd.d/rsh to set the flag "disable = no" and restarted xinetd. Following that, I’ve hashed two lines in /etc/pam.d/rsh :

auth required pam_securetty.so

auth required pam_rhosts_auth.so

As said, quick and dirty. It allowed rsh from vcs01-rhel as root, without password. Don’t try it at an unsercure environment, as it actually allows not only vcs01-rhel, but any and every computer on the net full rsh, password free, access to the server. Better think it over, right?

The first thing I’ve done after I’ve finished installing the software, was to undo my pam.d changes, and disable rsh service. Later, I will remove it.

So, we need to run the installer, which can be done by cding to /mnt/cdrom/rhel3_i686 and running ./installer -usersh

I’m asked about all the machines I need to install VCS on, I’m asked if I want to configure the cluster (which I do), and I set a name, a cluster ID (a number between 0 and 255). This Cluster ID is especially important when dealing with few Veritas Clusters running on the same infrastracture. If you have two clusters with the same Cluster ID, you get extra-large cluster, and a mess out of it. Mind it, if you’re ever into few clusters in one network.

I decide to set it up onwards. I decide to enable the web management GUI, and I decide to set IP for the cluster. This IP will be used for the resource group (called ClusterService by default), and will be a resource in it. When/if I have more resource groups, I should consider adding more IP addresses for them. At least one for each. In such a case, the cluster server is serving clients requests without them being aware of any "special" setup with the server, like, for example, it has switched over two times already.

I define heartbit networks. I’ve used eth1 as the private heartbit, and eth0 as both public network and "slow heartbit". I would add later some more virtual NICs to both nodes, and define them to be used as private heartbit as well.

Installing packages – I decide to install all optional packages. It’s not that I’m going to lack space. I did not install, mind you, VVM, because I want to simulate a no-volume-manager-enabled system. Just pure basic simple partitions.

Installation went fine, and I was happy as a puppy.

One thing to note – I wanted to install the Maintanance Pack I have. I was unable to eject my CD. running lsof | grep /mnt/cdrom revealed that CmdServer, some part of VCS, was using the cdrom, probably because, as root, I initiated the service from that location. I shut down vcs service, and started it again from another path, and I was able to eject my CD.

Installing the MP wasn’t that easy. The installer, much smarter this time, has required the package redhat-release which is a mandatory package in RedHat systems, but me, running Centos, had the package centos-release which just wouldn’t do the trick. I’ve decided to rebuild the package centos-release with an internal different name – redhat-release, and to do that, I’ve had to download the srpm of centos-release. You need to change the name and version so in your RPM you’ll have redhat-release-3ES-13.6.2. I’ve done it with this SRPM centos-release.spec file. Replace your centos-release srpm spec file with this one, and you should be just fine. Remove your current centos-release package, and you’ll be able to install your newly built (using rppbuild -bb centos-release.spec) redhat-release RPM (faked, of course). Mind you – it will overwrite your /etc/redhat-release, so you better back it up, just in case. I’ve take precaution, and restored the file to its fake RedHat contents. You can never know…

You could wonder why I haven’t used RHEL itself, but a clone, namely Centos. Although its for home usage only, the ease of updates, availability of packages (using yum) and the fact I do not want to steal software combined together bring me to install Centos for all my home usages. Production environments, however, it will be an official RedHat, I can gurantee that.

So, it’s installing MP2, which means removing some packages, and then installing newer versions. The reason they do not use "upgrade" option of RPM is beyond me, but so their nastiness about redhat-release version. So, if you’ve kept all the rules given here, you’re supposed to have VCS4.0 MP2 installed on your Linux Nodes. Good luck. Our next chapter would be installing and configuring Oracle DataBase on this setup. Stay tuned 🙂

When I’ve installed my lab’s SunCluster, only later did I install Veritas Volume Manager, aka VxVM. I believed they wouldn’t need it at my development dept., but I proved to be wrong. Only short while afterwards I’ve been asked to add VxVM to the configuration. I’ve added it, and installed it using Veritas install scripts. I’ve used vxsetup
to configure disks which were not the bootable disks, as I know the consequences of early development stages with hard-to-access volumes.

Now, a while later, I’ve been asked to encapsulate the bootable disk, and add it into VxVM.

After I’ve manually configured one node (and discovered SC global devices were encapsulated, but their did devices /etc/vfstab entries were not updated correctly), I’ve read /usr/cluster/bin/scvxinstall, and deduced how Sun do it. It appears they change the entry in /etc/vfstab to point to the real physical dev entry of the global device (for example, instead of /dev/did/dsk/d1s3, in my case, to /dev/dsk/c0t0d0s3). When set that way, encapsulation, using vxdiskadm, for example, can easily be done. Afterwards, the system can boot
correctly. To be on the safe side, After setting the encapsulation, I’ve rebooted the machine using the command "reboot — -x", so SunCluster and Veritas won’t have any possible collision. It worked, to my full surprise, and it’s now up and running!