Posts Tagged ‘scripting’

XenServer – Setting virtual disks names based on the VM names

Wednesday, January 2nd, 2013

One of the worst things you can have in XenServer, is some wize-guy performing a ‘forget storage’ on a storage device still holding virtual disks related to VMs. As XenServer database is internal (for the whole pool) and not per-VM, all references to this virtual disks disappear, and you remain with bunch of VMs without disks, and later on, when the recovered from the shock and restored the SR, with a bunch of virtual disks you have no clue as to where they belong. Why? Because we are lazy, and we tend to skip the part where you can (or is it – should?) define a custom name for your virtual disks so you would know later on (for example – in the case specified above) where they belong(ed).

To solve this annoying issue, and to save time for Citrix XenServer admins, I have created a script which resets the VDI (virtual disk object) names to the name of the VM+ the logical position of the virtual disk (example: xvda, hdb, etc), related to the VM. That way, it will become very easy to identify the disks in case of such annoying micro-catastrophy (micro because no data is lost, just where it belongs…).

The script can be called manually, and since we’re lazy people, and we will forget to handle it manually every said interval, and will accumulate virtual machines with “Template of XYZ” virtual disks, it can be called from cron. When called manually, it asks the user to proceed by pressing ‘Enter’. If called from cron, it just runs.

Enjoy!

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
#!/bin/bash
# This script will reset the names of the virtual disks used for each VM to a standard name, based on the VM name
# and the disk position
# It is meant to solve problems where due to 'forget storage' operations or the likes
# virtual disk associations disappear, and you face many disks having the same name
#
# Written by Ez-Aton: http://run.tournament.org.il
 
 
if [ -t 1 ]
then
        echo "This script will reset *all* VM disks to a name constructed of the VM and the disk name (xvda, hdb, etc)"
        echo "This operation is not reversible, however, it can be called repeatedly"
        echo "If you want this script to skip a said virtual disk, make sure its name includes the name of the VM"
        echo "For example 'vm1 the real important data disk' for a disk used by vm1."
        echo "Note that the name is case sensitive, and it is very important that to match the name using upper/lower case letters as needed"
        echo "To abort, press Ctrl+C"
        echo "To proceed, press Enter"
        read abc
fi
 
VM_LIST=`xe vm-list is-control-domain=false --minimal | tr , ' '`
 
for i in $VM_LIST
do
        # Resetting several parameters, so we have a clean start
        VM_NAME=""
        VBD_LIST=""
        VDI_LIST=""
        # We iterate through all existing VMs, to get both their names, and their disks
        VM_NAME="`xe vm-param-get uuid=$i param-name=name-label`"
        if [ -z "$VM_NAME" ]
        then
                # We have a problem with empty VM names, so we will use the VMs uuid
                VM_NAME=$i
        fi
        VBD_LIST=`xe vbd-list vm-uuid=$i --minimal | tr , ' '`
        for j in $VBD_LIST
        do
                # Resetting several parameters, so we have a clean start
                VDI_UUID=""
                DEV_NAME=""
                # We iterate through all existing VBDs to reset the VDI nane
                VDI_UUID=`xe vbd-param-get uuid=$j param-name=vdi-uuid`
                if [ "$VDI_UUID" == "<not in database>" ]
                then
                        # This is a virtual CDROM
                        continue
                fi
                DEV_NAME=`xe vbd-param-get uuid=$j param-name=device`
                VDI_NAME=`xe vbd-param-get uuid=$j param-name=vdi-name-label`
 
                # Test if the name was reset in the past or manually
                TGT_NAME="$VM_NAME $DEV_NAME"
                if [[ "$TGT_NAME" = "$VDI_NAME" ]]
                then
                        # There is nothing to do
                        echo "Name already includes VM name, so nothing to do"
                else
                        # Here we reset the VDI name
                        echo xe vdi-param-set uuid=$VDI_UUID name-label="$TGT_NAME"
                        xe vdi-param-set uuid=$VDI_UUID name-label="$TGT_NAME"
                fi
        done
done

Bonding + VLAN tagging + Bridge – updated

Wednesday, April 25th, 2012

In the past I hacked around a problem with the order of starting (and with several bugs) a network stack combined of network bonding (teaming) + VLAN tagging, and then with network bridging (aka – Xen bridges). This kind of setup is very useful for introducing VLAN networks to guest VMs. This works well on Xen (community, Server), however, on RHEL/Centos 5 versions, the startup scripts (ifup and ifup-eth) are buggy, and do not handle this operation correctly. It means that, depending on the update release you use, results might vary from “everything works” to “I get bridges without VLANs” to “I get VLANs without bridges”.

I have hacked a solution in the past, modifying /etc/sysconfig/network-scripts/ifup-eth and fixing some bugs in it, however, both maintaining the fix on every release of ‘initscripts’ package has proven, well, not to happen…

So, instead, I present you with a smarter solution, better adept to updates supplied from time to time by RedHat or Centos, using predefined ‘hooks’ in the ifup scripts.

Create the file /sbin/ifup-pre-local with the following contents:

 

#!/bin/bash
# $1 is the config file
# $2 is not interesting
# We will start the vlan bonding before any bridge
 
DIR=/etc/sysconfig/network-scripts
 
[ -z "$1" ] &amp;&amp; exit 0
. $1
 
if [ "${DEVICE%%[0-9]*}" == "xenbr" ]
then
    for device in $(LANG=C egrep -l "^[[:space:]]*BRIDGE=\"?${DEVICE}\"?" /etc/sysconfig/network-scripts/ifcfg-*) ; do
        /sbin/ifup $device
    done
fi

You can download this scrpit. Don’t forget to change it to be executable. It will call ifup for any parent device of xenbr* device called at. If the parent device is already up, no harm is done. If the parent device is not up, it will be brought up, and then the xenbr device can start normally.

Cables connection in Israel for Linux

Thursday, May 14th, 2009

Update to 0.2. Links remain the same. At the moment I cannot host many versions (it’s mostly uncomfortable), but this might change in the future.

I have created a GUI cables installer and configurator for L2TP on Linux.

I have noticed that there is no GUI solution, so, after this has been brought up, I have done it (!!!)

I have uploaded these files here, and you are welcome to use them.

Remember – they are designed for a blank Ubuntu (currently. More distros will be supported in the future, upon request) with not much of junk installed. Also – they are designed for the simple user. Double-click and run. That’s it.

Quoting my readme file:

L2TP Cables connection in Israel (and across the world, where relevant) by Ez-Aton

—About:
This is an installer and configurator for L2TP over cables in Israel
With some luck, by running this installer, you will be able to connect
to the Internet with a dialer!

The system assumes you have little technical knowledge of Linux and you
are not expected to have any. Follow the defaults, and you should be fine.

This configuration will be cross distro in the future, meaning it will work
both on your Ubuntu, your RHEL, your Centos, Mandrake, etc. In order for me
to be able to do so, please assist by sending information on systems I am
not familiar with yet, per the appendix at the bottom.
Also, you can feel free to send me info in case the system did not work for
you (and let me know what are the differences from a default installation),
or, as always, send me money.

Visit my technical blog for updates and all kind of other technical stuff, at

http://run.tournament.org.il

OSS work is meant to be based on others work, and that I have done. I would
like to thank (and mention below) the resources for without this would not
have happened.

I hope you enjoy this dialer!

Ez

—How to use
Simply double-click on the “cables” icon on your desktop, and the system will
get you connected.
For CLI utilization: Run /usr/local/bin/cables

—Tools and resources used:
To create this package I have used the following tools and resources
makeself http://megastep.org/makeself/
xl2tpd by http://www.xelerance.com/software/xl2tpd/
xl2tpd guide for Israel Cables http://stuff.pulkes.org/l2tp/
ISP LNS list http://www.cables.org.il/cable-vpn/vpn.html
My connect/disconnect scripts from http://run.tournament.org.il

—License
This package contents are under GNUv2 license, meaning you have full permission
to modify the contents of this package, except for the binary packages included
with it, where you are binded by their respective licenses.

—My Distro/ISP is not supported!
Well, these things happen. Over 300 distros our there, and I can’t have them all.
However – you have your own distro, right? For me to add it to this package
(assuming you don’t want to do this yourself) you will have to supply me with the
following info:
* What distro, kernel and version, and how you get the distro name
(for example – on Redhat – /etc/redhat-release. On Ubuntu – /etc/lsb-release)
* The file containing the version inforamtion (see above)
* The versions available from your repositories of xl2tpd or l2tpd for older
releases, and where you can get them
* Your ISP, your ISPs LNS names/addresses
* Your country
* All other info you think relevant

—Change log
0.2 – Added ability to enter manual LNS address. Added Orange LNS. Fixed fixroute to allow both IP and hostname without problems. Fixed cables connection script to run fixroute anyhow.
0.1 – Initial release

Download it here: cables_connect.sh

If you want the scripts and sources (not for the simple user!), you can get there here: l2tp-cables

HP EVA bug – Snapshot removed through sssu is still there

Friday, May 2nd, 2008

This is an interesting bug I have encountered:

The output of an sssu command should look like this:

EVA> DELETE STORAGE “\Virtual Disks\Linux\oracle\SNAP_ORACLE”

EVA>

It still leaves the snapshot (SNAP_ORACLE in this case) visible, until the web interface is used to press on “Ok”.

This happened to me on HP EVA with HP StorageWorks Command View EVA 7.0 build 17.

When sequential delete command is given, it looks like this:

EVA> DELETE STORAGE “\Virtual Disks\Linux\oracle\SNAP_ORACLE”

Error: Error cannot get object properties. [ Deletion completed]

EVA>

When this command is given for a non-existing snapshot, it looks like this:

EVA> DELETE STORAGE “\Virtual Disks\Linux\oracle\SNAP_ORACLE”

Error: \Virtual Disks\Linux\oracle\SNAP_ORACLE not found

So I run the removal command twice (scripted) on an sssu session without “halt_on_errors”. This removes the snapshots correctly.

Rotate Beryl/Compiz cube from command line

Tuesday, February 19th, 2008

We are about to have a stand in a show in Israel. To pull some attention, I have searched for a method to automate a random rotation of the famous Beryl/Compiz cube.

An extension of the method provided in here (using macros) is demonstrated below, using a script.

This is a bit more complicated, as I have used the motions of the mouse to achieve a “show” out of it (just changing desktops isn’t enough nowadays…)

Check out below for the full script.

#!/bin/bash
# This script will rotate the cube one click on each direction each predefined
# timer.
# Written by ezaton at tournament.org.il
# Check out my technical blog “Running Systems” at http://www.tournmament.org.il/run

# Set timer (seconds)
TIMER=10
VERT_TIMER=1

# Possible directions? 4 (up, down, left, right. will be marked from 0 to 3)
# Addition – set it to give higher priority to side-stepping. So max is 10, and only 0&1 represent
# up/down
# Added 10 to represent 2xleft and 11 to represent 2xright
POSS=12

# Temp command file
TMP_FILE=/tmp/rotate.macro

function create_file {
# This function will create and secure the macro file
\rm $TMP_FILE
if [ -f $TMP_FILE ]; then
echo “$TMP_FILE still exists”
exit 1
fi
echo “” > $TMP_FILE
chmod 700 $TMP_FILE
}

function run_macro {
# Run the actual macro
cat $TMP_FILE | xmacroplay $DISPLAY &>/dev/null
}

function left {
# This function will build the macro file for the “left” command
echo > $TMP_FILE
echo “MotionNotify 100 380″ >> $TMP_FILE
echo “KeyStrPress Alt_L” >> $TMP_FILE
echo “KeyStrPress Control_L” >> $TMP_FILE
echo “ButtonPress 1″ >> $TMP_FILE
echo “MotionNotify 100 380″ >> $TMP_FILE
echo “MotionNotify 130 380″ >> $TMP_FILE
echo “MotionNotify 150 380″ >> $TMP_FILE
echo “MotionNotify 170 380″ >> $TMP_FILE
echo “MotionNotify 190 380″ >> $TMP_FILE
echo “MotionNotify 210 380″ >> $TMP_FILE
echo “MotionNotify 230 380″ >> $TMP_FILE
echo “MotionNotify 250 380″ >> $TMP_FILE
echo “MotionNotify 270 380″ >> $TMP_FILE
echo “MotionNotify 290 380″ >> $TMP_FILE
echo “MotionNotify 310 380″ >> $TMP_FILE
echo “MotionNotify 330 380″ >> $TMP_FILE
echo “ButtonRelease 1″ >> $TMP_FILE
echo “KeyStrRelease Control_L” >> $TMP_FILE
echo “KeyStrRelease Alt_L” >> $TMP_FILE
run_macro
}

function up {
# This function will build the macro file for the “up” command
echo > $TMP_FILE
echo “MotionNotify 100 100″ >> $TMP_FILE
echo “KeyStrPress Alt_L” >> $TMP_FILE
echo “KeyStrPress Control_L” >> $TMP_FILE
echo “ButtonPress 1″ >> $TMP_FILE
echo “MotionNotify 100 100″ >> $TMP_FILE
echo “MotionNotify 100 120″ >> $TMP_FILE
echo “MotionNotify 100 140″ >> $TMP_FILE
echo “MotionNotify 100 160″ >> $TMP_FILE
echo “MotionNotify 100 180″ >> $TMP_FILE
echo “MotionNotify 100 200″ >> $TMP_FILE
echo “MotionNotify 100 220″ >> $TMP_FILE
echo “MotionNotify 100 240″ >> $TMP_FILE
echo “MotionNotify 100 260″ >> $TMP_FILE
echo “MotionNotify 100 280″ >> $TMP_FILE
echo “MotionNotify 100 300″ >> $TMP_FILE
echo “MotionNotify 100 320″ >> $TMP_FILE
echo “ButtonRelease 1″ >> $TMP_FILE
echo “KeyStrRelease Control_L” >> $TMP_FILE
echo “KeyStrRelease Alt_L” >> $TMP_FILE
run_macro
}

function right {
# This function will build the macro file for the “right” command
echo > $TMP_FILE
echo “MotionNotify 340 380″ >> $TMP_FILE
echo “KeyStrPress Alt_L” >> $TMP_FILE
echo “KeyStrPress Control_L” >> $TMP_FILE
echo “ButtonPress 1″ >> $TMP_FILE
echo “MotionNotify 340 380″ >> $TMP_FILE
echo “MotionNotify 320 380″ >> $TMP_FILE
echo “MotionNotify 300 380″ >> $TMP_FILE
echo “MotionNotify 280 380″ >> $TMP_FILE
echo “MotionNotify 260 380″ >> $TMP_FILE
echo “MotionNotify 240 380″ >> $TMP_FILE
echo “MotionNotify 220 380″ >> $TMP_FILE
echo “MotionNotify 200 380″ >> $TMP_FILE
echo “MotionNotify 180 380″ >> $TMP_FILE
echo “MotionNotify 160 380″ >> $TMP_FILE
echo “MotionNotify 140 380″ >> $TMP_FILE
echo “MotionNotify 120 380″ >> $TMP_FILE
echo “ButtonRelease 1″ >> $TMP_FILE
echo “KeyStrRelease Control_L” >> $TMP_FILE
echo “KeyStrRelease Alt_L” >> $TMP_FILE
run_macro
}

function down {
# This function will build the macro file for the “down” command
echo > $TMP_FILE
echo “MotionNotify 100 330″ >> $TMP_FILE
echo “KeyStrPress Alt_L” >> $TMP_FILE
echo “KeyStrPress Control_L” >> $TMP_FILE
echo “ButtonPress 1″ >> $TMP_FILE
echo “MotionNotify 100 330″ >> $TMP_FILE
echo “MotionNotify 100 310″ >> $TMP_FILE
echo “MotionNotify 100 290″ >> $TMP_FILE
echo “MotionNotify 100 270″ >> $TMP_FILE
echo “MotionNotify 100 250″ >> $TMP_FILE
echo “MotionNotify 100 230″ >> $TMP_FILE
echo “MotionNotify 100 210″ >> $TMP_FILE
echo “MotionNotify 100 190″ >> $TMP_FILE
echo “MotionNotify 100 170″ >> $TMP_FILE
echo “MotionNotify 100 150″ >> $TMP_FILE
echo “MotionNotify 100 130″ >> $TMP_FILE
echo “MotionNotify 100 110″ >> $TMP_FILE
echo “ButtonRelease 1″ >> $TMP_FILE
echo “KeyStrRelease Control_L” >> $TMP_FILE
echo “KeyStrRelease Alt_L” >> $TMP_FILE
run_macro
}

function fix_vert {
# Fixes a case of vertical extention (non-viewable screen)
sleep $VERT_TIMER
case “$1″ in
1) down
;;
0) up
;;
esac
}

# Verify we have xmacroplay
which xmacroplay
if [ "$?" -ne "0" ]; then
echo “Missing xmacroplay. Install it”
echo “Use apt get install xmacro”
exit 1
fi

# Do we use X and have a defined display? Won’t work otherwise…
if [ -z "$DISPLAY" ]; then
echo “DISPLAY is not defined. Exiting”
exit 1
fi

create_file

# We start where all is viewable

while true; do
# Select direction
DIRECTION=$RANDOM
let “DIRECTION %= $POSS”
# Debug:        echo “*** $DIRECTION ***”
case “$DIRECTION” in
0) up
fix_vert 1
;;
1) down
fix_vert 0
;;
[2-5]) left
;;
[6-9]) right
;;
10)     left
left
;;
11)     right
right
;;
esac
sleep $TIMER
done

exit 0

Quick provisioning of virtual machines

Friday, February 1st, 2008

When one wants to achieve fast provisioning of virtual machines, some solutions might come into account. The one I prefer uses Linux LVM snapshot capabilities to duplicate one working machine into few.

This can happen, of course, only if the host running VMware-Server is Linux.

LVM snapshots have one vast disadvantage – performance. When a block on the source of the snapshot is being changed for the first time, the original block is being replicated to each and every snapshot COOW space. It means that a creation of a 1GB file on a volume having ten snapshots means a total copy of 10GB of data across your disks. You cannot ignore this performance impact.

LVM2 has support for read/write snapshots. I have come up with a nice way of utilizing this capability to my benefit. An R/W snapshot which is being changed does not replicate its changes to any other snapshot. All changes are considered local to this snapshot, and are being maintained only in its COOW space. So adding a 1GB file to a snapshot has zero impact on the rest of the snapshots or volumes.

The idea is quite simple, and it works like this:

1. Create adequate logical volume with a given size (I used 9GB for my own purposes). The name of the LV in my case will be /dev/VGVM3/centos-base

2. Mount this LV on a directory, and create a VM inside it. In my case, it’s in /vmware/centos-base

3. Install the VM as the baseline for all your future VMs. If you might not want Apache on some of them, don’t install it on the baseline.

4. Install vmware-tools on the baseline.

5. Disable the service “kudzu”

6. Update as required

7. In my case I always use DHCP. You can set it to obtain its IP once from a given location, or whatever you feel like.

8. Shut down the VM.

9. In the VM’s .vmx file add a line like this:

uuid.action = “create”

I have added below (expand to read) two scripts which will create the snapshot, mount it and register it, including new MAC and UUID.

Press below for the scripts I have used to create and destroy VMs

create-replica.sh:

#!/bin/sh
# This script will replicate vms from a given (predefined) source to a new system
# Written by Ez-Aton, http://www.tournament.org.il/run
# Arguments: name

# FUNCITONS BE HERE
test_can_do () {
# To be able to snapshot, we need a set of things to happen
if [ -d $DIR/$TARGET ] ; then
echo “Directory already exists. You don’t want to do it…”
exit 1
fi
if [ -f $VG/$TARGET ] ; then
echo “Target snapshot exists”
exit 1
fi
if [ `vmrun list | grep -c $DIR/$SRC/$SRC.vmx` -gt "0" ] ; then
echo “Source VM is still running. Shut it down before proceeding”
exit 1
fi
if [ `vmware-cmd -l | grep -c $DIR/$TARGET/$SRC.vmx` -ne "0" ] ; then
echo “VM already registered. Unregister first”
exit 1
fi
}

do_snapshot () {
# Take the snapshot
lvcreate -s -n $TARGET -L $SNAPSIZE $VG/$SRC
RET=$?
if [ "$RET" -ne "0" ]; then
echo “Failed to create snapshot”
exit 1
fi
}

mount_snapshot () {
# This function creates the required directories and mounts the snapshot there
mkdir $DIR/$TARGET
mount $VG/$TARGET $DIR/$TARGET
RET=$?
if [ "$RET" -ne "0" ]; then
echo “Failed to mount snapshot”
exit 1
fi
}

alter_snap_vmx () {
# This function will alter the name in the VMX and make it the $TARGET name
cat $DIR/$TARGET/$SRC.vmx | grep -v “displayName” > $DIR/$TARGET/$TARGET.vmx
echo “displayName = \”$TARGET\”" >> $DIR/$TARGET/$TARGET.vmx
cat $DIR/$TARGET/$TARGET.vmx > $DIR/$TARGET/$SRC.vmx
\rm $DIR/$TARGET/$TARGET.vmx
}

register_vm () {
# This function will register the VM to VMWARE
vmware-cmd -s register $DIR/$TARGET/$SRC.vmx
}

# MAIN
if [ -z "$1" ]; then
echo “Arguments: The target name”
exit 1
fi

# Parameters:
SRC=centos-base         #The name of the source image, and the source dir
PREFIX=centos             #All targets will be created in the name centos-$NAME
DIR=/vmware               #My VMware VMs default dir
SNAPSIZE=6G              #My COOW space
VG=/dev/VGVM3           #The name of the VG
TARGET=”$PREFIX-$1″

test_can_do
do_snapshot
mount_snapshot
alter_snap_vmx
register_vm
exit 0

remove-replica.sh:

#!/bin/sh
# This script will remove a snapshot machine
# Written by Ez-Aton, http://www.tournament.org.il/run
# Arguments: machine name

#FUNCTIONS
does_it_exist () {
# Check if the described VM exists
if [ `vmware-cmd -l | grep -c $DIR/$TARGET/$SRC.vmx` -eq "0" ]; then
echo “No such VM”
exit 1
fi
if [ ! -e $VG/$TARGET ]; then
echo “There is no matching snapshot volume”
exit 1
fi
if [ `lvs $VG/$TARGET | awk '{print $5}' | grep -c $SRC` -eq "0" ]; then
echo “This is not a snapshot, or a snapshot of the wrong LV”
exit 1
fi
}

ask_a_thousand_times () {
# This function verifies that the right thing is actually done
echo “You are about to remove a virtual machine and an LVM. Details:”
echo “Machine name: $TARGET”
echo “Logical Volume: $VG/$TARGET”
echo -n “Are you sure? (y/N): “
read RES
if [ "$RES" != "Y" ]&&[ "$RES" != "y" ]; then
echo “Decided not to do it”
exit 0
fi
echo “”
echo “You have asked to remove this machine”
echo -n “Again: Are you sure? (y/N): “
read RES
if [ "$RES" != "Y" ]&&[ "$RES" != "y" ]; then
echo “Decided not to do it”
exit 0
fi
echo “Removing VM and snapshot”
}

shut_down_vm () {
# Shut down the VM and unregister it
vmware-cmd $DIR/$TARGET/$SRC.vmx stop hard
vmware-cmd -s unregister $DIR/$TARGET/$SRC.vmx
}

remove_snapshot () {
# Umount and remove the snapshot
umount $DIR/$TARGET
RET=$?
if [ "$RET" -ne "0" ]; then
echo “Cannot umount $DIR/$TARGET”
exit 1
fi
lvremove -f $VG/$TARGET
RET=$?
if [ "$RET" -ne "0" ]; then
echo “Cannot remove snapshot LV”
exit 1
fi
}

remove_dir () {
# Removes the mount point
rmdir $DIR/$TARGET
}

#MAIN
if [ -z "$1" ]; then
echo “No machine name. Exiting”
exit 1
fi

#PARAMETERS:
DIR=/vmware                #VMware default VMs location
VG=/dev/VGVM3            #The name of the VG
PREFIX=centos              #Prefix to the name. All these VMs will be called centos-$NAME
TARGET=”$PREFIX-$1″
SRC=centos-base           #The name of the baseline image, LVM, etc. All are the same

does_it_exist
ask_a_thousand_times
shut_down_vm
remove_snapshot
remove_dir

exit 0

Pros:

1. Very fast provisioning. It takes almost five seconds, and that’s because my server is somewhat loaded.

2. Dependable: KISS at its marvel.

3. Conservative on space

4. Conservative on I/O load (unlike the traditional use of LVM snapshot, as explained in the beginning of this section).

Cons:

1. Cannot streamline the contents of snapshot into the main image (LVM team will implement it in the future, I think)

2. Cannot take a snapshot of a snapshot (same as above)

3. If the COOW space of any of the snapshots is full (viewable through the command ‘lvs‘) then on boot, the source LV might not become active (confirmed RH4 bug, and this is the system I have used)

4. My script does not edit/alter /etc/fstab (I have decided it to be rather risky, and it was not worth the effort at this time)

5. My script does not check if there is enough available space in the VG. Not required, as it will fail if creation of LV will fail

You are most welcome to contribute any further changes done to this script. Please maintain my URL in the script if you decide to use it.

Thanks!

net-snmp broken in RHEL (and Centos, of course) – diskio

Saturday, June 9th, 2007

I’ve had a belief for quite a while now that Linux, unlike other types of systems, was unable to produce any I/O SNMP information. I only recently found out that it was partially true – all production-level distros, such as RedHat (and Centos, for that matter) were unable to produce any output for any SNMP DISKIO queries.

I had found a bugzilla entry about it, so I raise the glove in a request to any of the maintainers of an RH-compatible repositories to recompile (and maintain, of course) an alternate net-snmp package which supports diskio.

Meanwhile, I have found this blog post, which offers an alternate (and quite clumsy, yet working) solution to the disk performance measurement issue in Linux. I haven’t tried it yet, but I will, rather soon.

—Update—

I have used the script from the blog post mentioned above, and it works.

Speed could be an issue. Comparing two servers the speed differential was amazing.

Both servers are connected on the same switch as the server running the query is connected. Server1 has a P2 233MHz CPU, while Server2 has a dual 2.8GHz Xion CPU.

~$ time snmpwalk -c COMMUNITY -v2c Server1 1.3.6.1.4.1.2021.13.15 > /dev/null

real 0m0.311s
user 0m0.024s
sys 0m0.020s

~$ time snmpwalk -c COMMUNITY -v2c Server2 1.3.6.1.4.1.2021.13.15 > /dev/null

real 0m8.303s
user 0m0.044s
sys 0m0.012s

Looks like a huge difference. However, I believe it’s currently good enough for me.

Correction of a small but annoying error

Thursday, April 19th, 2007

For some reason (probably a typo) I’ve missed an important character in an example I gave here, but I have just recently fixed it. Anyhow, to clarify this, here is the extended description of the correction.

The $IFS Bash system variable defines what is the default separator between strings. Changing it can help when dealing with, for example, file names with spaces in them, variables which should be considered one unit, but are separated by semicolon, etc.

To change the default string separator from "space or tab or new-line" to new-line only. you need to set, in Bash the following parameter:

IFS=$’\n’

HP-UX, Oracle 8i, DataProtector, libobk.sl

Tuesday, July 11th, 2006

Imagine Omniback 4.x to Omniback 5.5 upgrade on HP-UX. Imagine you assume all existing backup procedure (and you were told there is license only for disk agent) is based on filesystem backup. Assume you know there’s Oracle installed on this server, but no relevant agent (again, filesystem only, no DB agents…)

You are cautious. You move the current /opt/omni to /opt/omni.old directory. You hash the line containing the relevant entry in /etc/inetd.conf. You are prepared.

You install the newer version by running the installer script with the flag -install da, so you would install only Disk Agent (after all, this is nothing but a client to this whole backup procedure).

You check everything, and it all seems to work correctly as far as you care or know. Suddenly, someone notices that Oracle Listener (TNS) does not listen anymore. Trying to bring it back up results in a message which seems like this:

/usr/lib/pa20_64/dld.sl: Unable to find library ‘libobk.sl’.

It doesn’t look good. You are in a little crisis. Restart to the Oracle Engine itself results in a shutdown, but it never starts back again. It doesn’t look good.

It appears that there was an Oracle Agent for Omniback installed there previously, and that you removed it uncleanly by your Disk-Agent-Only upgrade.

The solution could have been to install the Oracle Agent. It can be also related to recreating the required links, say from /opt/omni.old/lib/libbo2oracle8_64.sl or from $ORACLE_HOME/lib64/libobk.sl.backup (if there were any…) to $ORACLE_HOME/lib64/libobk.sl, the first is based on the older Omniback, the later is based on the assumption a backup was made.

The quickest solution was to install the newer agent, with the Oracle8i agent (called “oracle8″) and link: “ln -s /opt/omni/lib/libob2oracle8_64.sl ~oracle/lib64/libobk.sl” (assuming we’re running Oracle 8i on 64bit HP-UX PA-RISC).

Multihomed routing (split access load balancing) and OpenVPN

Sunday, June 25th, 2006

We have one connection via ATM like interface and we have one PPP connection via xDSL (described here), and we want load balancing for this whole party.

Following this specific part of lartc.org guide, we’ve managed to get this to work. The idea goes like this (Centos 4.3):

1. Do not state default route for the machine. Not in /etc/sysconfig/network and not in /etc/sysconfig/network-scripts/ifcfg-ethX

2. Using adsl-setup, we’ve defined our ADSL connection. Verify you have an entry DEFROUTE=no in your /etc/sysconfig/network-scripts/ifcfg-ppp0

3. find a way to start the following script after your network interfaces are up. I assume, in this script, that your ATM interface is eth1. multiroute.txt

The reason for specifically stating SERVER is that our DNS server requires recursive DNS for its settings, and I can use my ISP’s DNS Server only when using the corresponding link. Since both links are for different ISPs, I need to “bind” SERVER to a specific route.

Note that this solution is only temporary. At the moment, it is far from being complete, and many tests should be done yet, before I can call it a working solution. I might combine it with /etc/ppp/ip-up.local script, or I might add it as a seperated service in /etc/init.d, which would start after all interfaces are up and running. Not final yet.

With all this working like charm, we’ve had a huge issue – our OpenVPN server, which worked correctly just until then failed to work smoothly. Sometimes clients were able to connect, and sometimes they were unable to do so…

I got the following error message in my logs: “x.y.z.m:2839 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)

The cause, as it seemed to me, was that OpenVPN’s UDP packets were routed via alternate route for each target client. Being UDP, they were not part of an active session, but were stateless, which resulted in a different routing descision each time they were directed at the OpenVPN client. I’ve searched for it, although I was not optimistic, because multihomed routing, with multiple ways out wasn’t very common. I was suprised to find this post, with it’s follow-up, which dealt exactly with my case.

Since I cannot bind it to an internal IP address (although I’ve tried – it didn’t work), I will test TCP based configuration tomorrow morning.

===============================================================================

Update

===============================================================================

I don’t usually update posts but add new posts with links. However, in this case it was important enough for me to update this hot topic so I’ve decided to just add the new stuff.

First – I’ve failed. Since I do not have too much time here, I did not feel confident to leave a system yet untested. Especially when such a router is an essential link in this company.

I’ve tried using TCP based connection, but, still again, one client was able to connect, while the 2nd one did so for only a short while, and failed maintaining a working connection. I went back to UDP…

I came up with the following idea – if I can use some sort of tagging to differentiate the UDP packets sourced at the router, at the OpenVPN application, I could try and set a routing rule which will force them into a specific routing chain, and force them through my interface.

It didn’t work quite well. I was able to do the followin trick, but for no avail:

iptables -t mangle -A OUTPUT -p udp –sport 5001 -j MARK –set-mark 1

and then, using “ip” command:

ip rule add fwmark 1 table T1

which should have redirected all outbound UDP with source port 5001 (this is the one I use for my OpenVPN, due to legacy considerations), to the T1 routing table – a table directed outside with default route via eth1.

I don’t know why it failed. Almost seemed to work, but no…

I returned the system to a single-path setup, with PPP0 only acting as a manual alternate path in case where the primary path is down. Would work for now.