Posts Tagged ‘Apache’

Some few small insights

Wednesday, March 12th, 2008

Lately I have been overloaded above my capabilities. This did not prevent me from doing all kind of things, but most of them are too small to justify a real entry here, so I have decided to make a small collection of small stuff someone might need to know, in order to make it indexed in search engines. These small insights might save some time for someone. This is a noble cause.

1. Oreon is a nice overlay for Nagios, however, it is poorly documented, and some of the existing docs are in French. I have put hours on building it into a working setup, and I hope to be able to write down the process as is.

2. “Sun Java System Active Server Pages” does not support 64bit Linux installations – at least not if you’re interested in using it with your existing Apache server. Look here. Seems nothing has changed.

3. Under Ubuntu 7.10, Compiz suffers from a major memory leak when using NVidia display adapters. You can read about it in the bug page. I was able, thanks to this link, to workaround it using compiz –indirect-rendering . Does not see to cause any ill-effect on my display performance.

4. Suse 10 and wireless cards – This one is a great guide, which I would happily recommend.

5. Flushing the existing read buffer for your Linux machine (should never be done, unless you’re testing performance) can be done by running the following command:

echo 1 > /proc/sys/vm/drop_caches

Seems to be enough for today. Hope these tips help.

Quick provisioning of virtual machines

Friday, February 1st, 2008

When one wants to achieve fast provisioning of virtual machines, some solutions might come into account. The one I prefer uses Linux LVM snapshot capabilities to duplicate one working machine into few.

This can happen, of course, only if the host running VMware-Server is Linux.

LVM snapshots have one vast disadvantage – performance. When a block on the source of the snapshot is being changed for the first time, the original block is being replicated to each and every snapshot COOW space. It means that a creation of a 1GB file on a volume having ten snapshots means a total copy of 10GB of data across your disks. You cannot ignore this performance impact.

LVM2 has support for read/write snapshots. I have come up with a nice way of utilizing this capability to my benefit. An R/W snapshot which is being changed does not replicate its changes to any other snapshot. All changes are considered local to this snapshot, and are being maintained only in its COOW space. So adding a 1GB file to a snapshot has zero impact on the rest of the snapshots or volumes.

The idea is quite simple, and it works like this:

1. Create adequate logical volume with a given size (I used 9GB for my own purposes). The name of the LV in my case will be /dev/VGVM3/centos-base

2. Mount this LV on a directory, and create a VM inside it. In my case, it’s in /vmware/centos-base

3. Install the VM as the baseline for all your future VMs. If you might not want Apache on some of them, don’t install it on the baseline.

4. Install vmware-tools on the baseline.

5. Disable the service “kudzu”

6. Update as required

7. In my case I always use DHCP. You can set it to obtain its IP once from a given location, or whatever you feel like.

8. Shut down the VM.

9. In the VM’s .vmx file add a line like this:

uuid.action = “create”

I have added below (expand to read) two scripts which will create the snapshot, mount it and register it, including new MAC and UUID.

Press below for the scripts I have used to create and destroy VMs

create-replica.sh:

#!/bin/sh
# This script will replicate vms from a given (predefined) source to a new system
# Written by Ez-Aton, http://www.tournament.org.il/run
# Arguments: name

# FUNCITONS BE HERE
test_can_do () {
# To be able to snapshot, we need a set of things to happen
if [ -d $DIR/$TARGET ] ; then
echo “Directory already exists. You don’t want to do it…”
exit 1
fi
if [ -f $VG/$TARGET ] ; then
echo “Target snapshot exists”
exit 1
fi
if [ `vmrun list | grep -c $DIR/$SRC/$SRC.vmx` -gt “0” ] ; then
echo “Source VM is still running. Shut it down before proceeding”
exit 1
fi
if [ `vmware-cmd -l | grep -c $DIR/$TARGET/$SRC.vmx` -ne “0” ] ; then
echo “VM already registered. Unregister first”
exit 1
fi
}

do_snapshot () {
# Take the snapshot
lvcreate -s -n $TARGET -L $SNAPSIZE $VG/$SRC
RET=$?
if [ “$RET” -ne “0” ]; then
echo “Failed to create snapshot”
exit 1
fi
}

mount_snapshot () {
# This function creates the required directories and mounts the snapshot there
mkdir $DIR/$TARGET
mount $VG/$TARGET $DIR/$TARGET
RET=$?
if [ “$RET” -ne “0” ]; then
echo “Failed to mount snapshot”
exit 1
fi
}

alter_snap_vmx () {
# This function will alter the name in the VMX and make it the $TARGET name
cat $DIR/$TARGET/$SRC.vmx | grep -v “displayName” > $DIR/$TARGET/$TARGET.vmx
echo “displayName = “$TARGET”” >> $DIR/$TARGET/$TARGET.vmx
cat $DIR/$TARGET/$TARGET.vmx > $DIR/$TARGET/$SRC.vmx
rm $DIR/$TARGET/$TARGET.vmx
}

register_vm () {
# This function will register the VM to VMWARE
vmware-cmd -s register $DIR/$TARGET/$SRC.vmx
}

# MAIN
if [ -z “$1” ]; then
echo “Arguments: The target name”
exit 1
fi

# Parameters:
SRC=centos-base         #The name of the source image, and the source dir
PREFIX=centos             #All targets will be created in the name centos-$NAME
DIR=/vmware               #My VMware VMs default dir
SNAPSIZE=6G              #My COOW space
VG=/dev/VGVM3           #The name of the VG
TARGET=”$PREFIX-$1″

test_can_do
do_snapshot
mount_snapshot
alter_snap_vmx
register_vm
exit 0

remove-replica.sh:

#!/bin/sh
# This script will remove a snapshot machine
# Written by Ez-Aton, http://www.tournament.org.il/run
# Arguments: machine name

#FUNCTIONS
does_it_exist () {
# Check if the described VM exists
if [ `vmware-cmd -l | grep -c $DIR/$TARGET/$SRC.vmx` -eq “0” ]; then
echo “No such VM”
exit 1
fi
if [ ! -e $VG/$TARGET ]; then
echo “There is no matching snapshot volume”
exit 1
fi
if [ `lvs $VG/$TARGET | awk ‘{print $5}’ | grep -c $SRC` -eq “0” ]; then
echo “This is not a snapshot, or a snapshot of the wrong LV”
exit 1
fi
}

ask_a_thousand_times () {
# This function verifies that the right thing is actually done
echo “You are about to remove a virtual machine and an LVM. Details:”
echo “Machine name: $TARGET”
echo “Logical Volume: $VG/$TARGET”
echo -n “Are you sure? (y/N): “
read RES
if [ “$RES” != “Y” ]&&[ “$RES” != “y” ]; then
echo “Decided not to do it”
exit 0
fi
echo “”
echo “You have asked to remove this machine”
echo -n “Again: Are you sure? (y/N): “
read RES
if [ “$RES” != “Y” ]&&[ “$RES” != “y” ]; then
echo “Decided not to do it”
exit 0
fi
echo “Removing VM and snapshot”
}

shut_down_vm () {
# Shut down the VM and unregister it
vmware-cmd $DIR/$TARGET/$SRC.vmx stop hard
vmware-cmd -s unregister $DIR/$TARGET/$SRC.vmx
}

remove_snapshot () {
# Umount and remove the snapshot
umount $DIR/$TARGET
RET=$?
if [ “$RET” -ne “0” ]; then
echo “Cannot umount $DIR/$TARGET”
exit 1
fi
lvremove -f $VG/$TARGET
RET=$?
if [ “$RET” -ne “0” ]; then
echo “Cannot remove snapshot LV”
exit 1
fi
}

remove_dir () {
# Removes the mount point
rmdir $DIR/$TARGET
}

#MAIN
if [ -z “$1” ]; then
echo “No machine name. Exiting”
exit 1
fi

#PARAMETERS:
DIR=/vmware                #VMware default VMs location
VG=/dev/VGVM3            #The name of the VG
PREFIX=centos              #Prefix to the name. All these VMs will be called centos-$NAME
TARGET=”$PREFIX-$1″
SRC=centos-base           #The name of the baseline image, LVM, etc. All are the same

does_it_exist
ask_a_thousand_times
shut_down_vm
remove_snapshot
remove_dir

exit 0

Pros:

1. Very fast provisioning. It takes almost five seconds, and that’s because my server is somewhat loaded.

2. Dependable: KISS at its marvel.

3. Conservative on space

4. Conservative on I/O load (unlike the traditional use of LVM snapshot, as explained in the beginning of this section).

Cons:

1. Cannot streamline the contents of snapshot into the main image (LVM team will implement it in the future, I think)

2. Cannot take a snapshot of a snapshot (same as above)

3. If the COOW space of any of the snapshots is full (viewable through the command ‘lvs‘) then on boot, the source LV might not become active (confirmed RH4 bug, and this is the system I have used)

4. My script does not edit/alter /etc/fstab (I have decided it to be rather risky, and it was not worth the effort at this time)

5. My script does not check if there is enough available space in the VG. Not required, as it will fail if creation of LV will fail

You are most welcome to contribute any further changes done to this script. Please maintain my URL in the script if you decide to use it.

Thanks!

NabRSS updated to version 0.2

Saturday, July 14th, 2007

NabRSS has been updated from the version posted here.

NabRSS is a set of PHP scripts meant to assist anyone with a PHP/MySQL/Apache server available to gather feeds from RSS sources and send them via Nabaztag API to his/her Nabaztag.

Available here: nabrss-0.2.tar.gz

Quoting from the changelog:

NabRSS Version 0.2 Jul 14th 07:
– get.php: Fixed an incorrect condition in function debug_set
– get.php: Added function set_ear_pos to randomize ear position on every post
– get.php: Added function set_voice to randomize each RSS post voice
– list.php: Added a “Reset” button to allow reseting the last post saved in the date field inside the DB

Todo (short range):
– DB: Extend the scheme and move voices into DB as a seperated table
– add_url.php: Select a voice for each RSS feed. Possiblity to select “random”
– list.php: Show selected voice for each RSS feed
– get.php: Insert a function to deal with voices per post

NabRSS initial release (0.1)

Friday, July 13th, 2007

I have written an RSS to Nabaztag (using API) php scripts. Special thanks to magpierss, for doing the complicated work of converting RSS feeds to PHP-usable arrays.

This system requires MySQL server, Apache server and PHP version 5 (haven’t tested it on PHP version 4).

The scripts are supplied under GPL license, and are free for use and distribution. Please keep the header information containing my details on every distribution of the scripts. Also, please help me make the scripts better, by leaving feedback and sending me fixes.

You can find the archive here: nabrss-0.1.tar.gz

Enjoy!

Subversion (SVN) over SSL in 10 small steps

Friday, April 27th, 2007

I have installed SVN on Centos4 (RHEL4) following these small and short steps.

1. Check out svnbook. This is the place where all your later questions will be answered.

2. Using Centos/RHEL and still not using Dag Wieers and RPMForge’s RPM repository?

3. Install using YUM or Apt the following packages:

mod_dav_svn

subversion

mod_ssl

distcache

neon

4. Create your SVN repository root directory and cd to there: "mkdir /var/www/svn ; cd /var/www/svn"

5. Create your new repository. I’ve used the name "projects" and I maintain the name later on. If you decide on another name, make sure to change wherever valid later: "svnadmin create projects"

6. Change ownership to Apache: "chown -R apache.apache projects"

7. Rename /etc/httpd/conf.d/subversion.conf to /etc/httpd/conf.d/subversion.conf.old

8. Edit /etc/httpd/conf/ssl.conf and add the following lines:

–Near the beginning, add these two lines:

LoadModule dav_svn_module modules/mod_dav_svn.so
LoadModule authz_svn_module modules/mod_authz_svn.so

–Just above the line saying </VirtualHost> enter the following:

<Location />

      DAV svn
      SVNParentPath /var/www/svn
      SSLRequireSSL
      AuthType Basic
      AuthName "Company"
      AuthUserFile /etc/httpd/conf.d/passwd
      Require valid-user

</Location>

9. Check Apache to verify you didn’t destroy anything. No need, by the way, to create SSL self-signed certificate, as the installation of mod_ssl creates one for you: "apachectl -t". If you get "Ok", you’re Ok.

10. Add users by using the utility "htpasswd". The first time will require the flag "-c" which tells htpasswd to create the file. Later on, no need for this flage. Exmaple: The first user will require: "htpasswd -c /etc/httpd/conf.d/passwd user1". The 2nd user will require just "htpasswd /etc/httpd/conf.d/passwd user2".

Done. All you need to do is point your SVN client to https://<IP or Name>/projects and you’re good to go (of course – as soon as you specify your username and password).