Poor man’s load balancing
I was requested to setup a “poor man’s” load balancing. The server accesses various HTTP servers with a GET command, and the customer fears that the server’s IP will get blocked. To work around this, or at least – to minimize the problem, the customer has purchased three IP addresses.
I have assigned all three addresses to the server, and was into smart routing as a solution. I did not want to capture all outbound communication, but only HTTP (port 80). Also – the system is Centos, which means there are only few available iptbles modules, so “random” module is not an option. Compiling a new kernel for a server across an ocean didn’t sound like the best idea, so I have attempted to work with the available tools.
With net.ipv4.conf.default.rp_filter set to 0 in /etc/sysctl.conf , and with real internet IPs (won’t work above NAT), I added the following rules to the mangle table:
iptables -t mangle -N CUST_OUTPUT
iptables -t mangle -A CUST_OUTPUT -o ! lo -p tcp -m recent –remove -j MARK –set-mark 1
iptables -t mangle -A OUTPUT -o ! lo -p tcp –dport 80 -m state –state NEW -m recent –update –seconds 60 –hitcount 6 -j CUST_OUTPUT
iptables -t mangle -A OUTPUT -o ! lo -p tcp –dport 80 -m state –state NEW -m recent –update –seconds 60 –hitcount 4 -j MARK –set-mark 3
iptables -t mangle -A OUTPUT -m state –state NEW -m mark –mark 3 -j ACCEPT
iptables -t mangle -A OUTPUT -o ! lo -p tcp –dport 80 -m state –state NEW -m recent –update –seconds 60 –hitcount 2 -j MARK –set-mark 2
iptables -t mangle -A OUTPUT -m state –state NEW -m mark –mark 2 -j ACCEPT
iptables -t mangle -A OUTPUT -o ! lo -p tcp –dport 80 -m state –state NEW -m recent –set
To the nat table, I have added these following rules:
iptables -t nat -A POSTROUTING -m mark –mark 2 -j SNAT –to 1.1.1.2
iptables -t nat -A POSTROUTING -m mark –mark 3 -j SNAT –to 1.1.1.3
iptables -t nat -A POSTROUTING -m mark –mark 1 -j SNAT –to 2.3.4.5
(replaced the real customer’s IPs with the junk above). Of course – all three IP addresses are accessible from the Internet and fully routable.
After a short run, I saw the following lines when running iptables -t nat -L -v
Chain POSTROUTING (policy ACCEPT 39055 packets, 2495K bytes)
pkts bytes target prot opt in out source destination
143 8580 SNAT all — any any anywhere anywhere MARK match 0x2 to:1.1.1.2
144 8640 SNAT all — any any anywhere anywhere MARK match 0x3 to:1.1.1.3
72 4320 SNAT all — any any anywhere anywhere MARK match 0x1 to:2.3.4.5
Statistically, this is quite ok. Mark 0x1 forces the same route as Mark 0x0, so this is rather balanced.
Works like a charm, for now 🙂