Posts Tagged ‘bonding’

Xen Networking – Bonding with VLAN Tagging

Thursday, October 23rd, 2008

The simple scripts in /etc/xen/scripts which manage networking are fine for most usages, however, when your server is using bonding together with VLAN tagging (802.11q) you should consider an alternative.

A PDF document written by Mark Nielsen, GPS Senior Consultant, Red Hat, Inc (I lost the original link, sorry) named “BOND/VLAN/Xen Network Configuration” as a service to the community, game me few insights on the subject. Following one of its references, I saw a bit more elegant method of doing a bridging setup under RedHat, which takes managing the bridges away from xend, and leaves it at the system level. Lets see how it’s done on RedHat style Linux distribution.

Manage your normal networking configurations

If you’re using VLAN tagging over bonding, than you should have to setup a bonding device (be it bond0) which has definitions such as this:

/etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
ISALIAS=no

/etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
BOOTPROTO=none
BONDING_OPTS=”mode=1 miimon=100″
ONBOOT=yes

This is rather stright-forward, and should be rather a default for such a setup. Now comes the more interesting part. Originally, the next configuration part would be bond0.2 and bond0.3 (in my example). The original configuration would have looked like this (this is in bold because I tend to fast-read myself, and tend to miss things too often). This is not how it should look when we’re done!

/etc/sysconfig/network-scripts/ifcfg-bond0.2 (same applies to ifcfg-bond0.3)

DEVICE=bond0.2
BOOTPROTO=static
IPADDR=192.168.0.2
NETMASK=255.255.255.0
ONBOOT=yes
VLAN=yes

Configure bridging

To setup a bridge device for bond0.2, replace the mentioned above ifcfg-bond0.2 with this new /etc/sysconfig/network-scripts/ifcfg-bond0.2

DEVICE=bond0.2
BOOTPROTO=static
ONBOOT=yes
VLAN=yes
BRIDGE=xenbr0

Now, create a new file /etc/sysconfig/network-scripts/ifcfg-xenbr0

DEVICE=xenbr0
BOOTPROTO=static
IPADDR=192.168.0.2
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=bridge

Now, on network restart, the bridge will be brought up, holding the right IP address – all done by initscripts, with no Xen intervention. You will want to repeat the last the “Configure bridge” part for any additional bridge you want to be enabled for Xen machines.

Don’t let Xen bring any bridges up

This is the last part of our drill, and it is very important. If you don’t do it, you’ll get a nice networking mess. As said before, Xen (community), by default, can’t handle bondings or vlan tags, so it will attempt to create or modify bridges to eth0 or the likes. Edit /etc/xen/xend-config.sxp and remark any line containing a directive containing starting with “network-script“. Such a directive would be, for example

(network-script network-bridge)

Restart xend and restart networking. You should now be able to configure VMs to use xenbr0 and xenbr1, etc (according to your own personal settings).

Bonding in RedHat RHEL4

Sunday, July 20th, 2008

This is a rather common knowledge now that on RHEL4 you need to state inside /etc/modprobe.conf the following line, when you want more than one bonding interfaces:

options bonding max_bonds=2

Then you attempt to use a trick to address different bonding devices with their name (aka, bond0 and bond1, and maybe bond2, etc), using an option as follows in your /etc/modprobe.conf:

options bond1 -o bond1 miimon=100

It works perfectly fine, until you try to set different parameters for your bonding devices, such as in this example (again, from /etc/modprobe.conf):

options bond0 -o bond0 mode=1 miimon=100
options bond1 -o bond1 mode=1 arp_validate=1 arp_ip_target=1.2.3.4 arp_interval=1000

These different options will not work. The 2nd (and all next) bonding devices will use bond0’s settings.

A note about this can be found in /usr/share/doc/kernel-doc-2.6.9/Documentation/networking/bonding.txt (requires the package “kernel-doc”):

NOTE: It has been observed that some Red Hat supplied kernels are apparently unable to rename modules at load time (the “-o bond1” part).  Attempts to pass that option to modprobe will produce an “Operation not permitted” error.  This has been reported on some Fedora Core kernels, and has been seen on RHEL 4 as well.  On kernels exhibiting this problem, it will be impossible to configure multiple bonds with differing parameters.

Without the ability to rename modules, we are unable to set, through /etc/modprobe.conf any bond-specific options.

An option which cannot be found in /usr/share/doc/initscripts-7.93.31.EL/sysconfig.txt (part of the “initscripts” package) is to remove any bond-specific parameters from /etc/modprobe.conf and to add to /etc/sysconfig/network-scripts/ifcfg-bondX a line as follows:

BONDING_MODULE_OPTS=’miimon=100 primary=eth0′

Here you can state your bonding options, and when you will restart your networking (provided you actually unload the “bonding” module during that process), your bonds will behave as you expect them to.

A small thing I need to confirm yet is the behavior of the bonding device if settings are changed without unloading the “bonding” module between the ifdown and ifup commands.

Vlan Tagging with bonding network interface on RHEL4

Saturday, May 17th, 2008

This is not a simple task, as there are few things which should actually happen for it to work.

First – the switch port should support vlan tagging (of course, right?)

I have used vlan2 for “external” network, and vlan3 for “internal” network.

My configuration looks like this:

ifcfg-eth0:

DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
ISALIAS=no

ifcfg-eth1:

DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
ISALIAS=no

ifcfg-bond0:

DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes

ifcfg-bond0.2:

DEVICE=bond0.2
BOOTPROTO=static
IPADDR=1.2.3.4
NETMASK=255.255.255.0
ONBOOT=yes
VLAN=yes

ifcfg-bond0.3:

DEVICE=bond0.3
BOOTPROTO=static
IPADDR=192.168.0.1
NETMASK=255.255.255.0
ONBOOT=yes
VLAN=yes

I hope it helps anyone who is into vlan tagging over bonding interfaces.