Running Systems

The idea in general is to have multiple wireless networks at home – one for the house residents, the other for visitors. The home network should have full access to everything, while the guest network should be able to reach the Internet, but nothing else.

I have Asus RT-AC87U, which is a fine router, but does not show these capabilities in its web GUI. I had flushed it with a derived firmware called AsusWRT-Merlin which added the ability to insert custom scripts.

I’ve had to research a bit, until I got something working. For future tinkering, and for any who requires it, I will add my scripts here.

First – in the web interface, enable guest network and, under Administration->System enable JFFS custom scripts.

Then, connect via SSH to the router, and place a script called /jffs/scripts/services-start containing:

#!/bin/sh
touch /tmp/000brstarted
PATH=”/sbin:/usr/sbin:/bin:/usr/bin:${PATH}”
robocfg vlan 100 ports “1t 2t 3t 4t 5t 8t”
vconfig add eth0 100
ifconfig vlan100 up
brctl addbr br1
brctl addif br1 vlan100
brctl delif br0 wl0.1
brctl addif br1 wl0.1
ifconfig br1 192.168.230.254 netmask 255.255.255.0 up
nvram set lan_ifnames=”vlan1 eth1″
nvram set lan_ifname=”br0″
nvram set lan1_ifnames=”vlan100 wl0.1″
nvram set lan1_ifname=”br1″
nvram set lan1_ipaddr=192.168.230.254
nvram commit
killall eapd
eapd

Run chmod +x /jffs/scripts/services-start so that it will work correctly.

This script will configure VLAN100 on all ports (including the internal ones 5 and 8), as VLAN tags (meaning – not access). Then it will add the VLAN to eth0 – which is the host interface for the external switch ports (eth1 is for the Wireless ports), bring it up, and create a bridge consisting of vlan100 and the additional wireless sub-interface wl0.1 (which is the guest interface). I did not bother setting up 5GHz guest network, so I didn’t have an additional wl1.1 sub-interface. If you configure a 5GHz guest network, you will need to add it to the bridge device. Then I’ve given the bridge interface an IP address so I could test it from my router, and setup nvram to hold these settings. Unfortunately, these settings must be defined each boot, and they are not kept without the script.

Maybe on my next post I will describe my switch network layout and settings. On a future post, I might even describe how to transfer VLANs to a VM running under KVM, and maybe even explain my router settings, so that eventually the readers (other than myself, of course) could reproduce this setup at their homes.

I use this blog as an external memory. I found myself lately looking for the obvious patches for Oracle GI and RDBMS products, and although I eventually reach the right location, the time consumed looking for them is a wasted time. So – to make sure I can remember them correctly, here are the two important sites:

Oracle OPatch: https://updates.oracle.com/download/6880880.html

Oracle GI/DB patches: https://support.oracle.com/epmos/faces/DocumentDisplay?id=2118136.2

(This article is the essence of a post from this Redhat Archive and it goes as follows:

Problem: You need to detect what deletes files on your Linux

Solution: Using auditd, with the right flags, you could get a lot of information.

In Practice:

• If the mount point/directory is /oracle, then:
• (as root:) auditctl –w /oracle -k whodeletedit -p w
  (Explanation: Monitor the directory /oracle, and log everything under the label “whodeleteit”. Monitor write operations)
• To see, later, who deleted files, run (as root): ausearch -i -k whodeletedit -x /bin/rm
• You would want to stop the logging as soon as you found the culprit, by running (as root):  auditctl –W /oracle -k whodeletedit -p w

I hope it helps you just as it helps me.

Assume a server has two network interfaces as follows:

• eth0 : 192.168.0.1/24
• eth1 : 192.168.10.1/24

Let’s assume these interfaces reside on the different VLANs. Lets assume they were connected incorrectly, in such a way that eth0 is connected to VLAN 10, which servers 192.168.10.0/24 and eth1 is connected to VLAN 2, which serves 192.168.0.0/24.

You would expect that queries by other hosts on VLAN 2 (which is connected to eth1, but serves 192.168.0.0/24!) would not get responses from the server. You are wrong.

Linux will answer who-is queries on VLAN 2, replying with eth1’s MAC address to queries for 192.168.0.1 IP address.

This example is a simple example, but it can get ugly if your eth0 mimics a different network, and you want the server to be disconnected. I have had to “forge” a network setup on a different VLAN, mimicking the original network and subnet. However – a “backdoor” I have opened (on an additional NIC) between the mimicking server and the original server on a different IP class (a private one) resulted in the mimicking server answering to ARP queries, causing the clients to attempt connecting to the mimicking server instead of the production server. The clients could not complete the TCP handcheck because the mimicking server attempted to contact them via eth0, which was on the fake network, and did not actually reach anywhere.

This was a more complex example, however – the result is the same – the response on the “wrong” network interface to ARP who-is queries might hijack data which should be delivered elsewhere.

There is a solution! You need to setup the sysctl parameter arp_ignore  to either of the following values. The parameter is hidden in /proc/sys/net/ipv4/conf/<NIC>/arp_ignore

The parameters documentation is as follows:

arp_ignore – INTEGER
Define different modes for sending replies in response to received ARP requests that resolve local target IP addresses:
0 – (default): reply for any local target IP address, configured on any interface
1 – reply only if the target IP address is local address configured on the incoming interface
2 – reply only if the target IP address is local address configured on the incoming interface and both with the sender’s IP address are part from same subnet on this interface
3 – do not reply for local addresses configured with scope host, only resolutions for global and link addresses are replied

The value “1” or “2” would do the trick in such cases.

As far as routing goes, it is a very bad idea to place multiple network interfaces with IPs of a single network (subnet). The routing table, which decides which interface to route the data through, reads the table line by line, thus – all traffic goes through a single interface of the batch (-> of the interfaces “living” on the same network). A common way of working around it is by using network teaming (Linux = bonding) to handle round-robin or active/passive of multiple interfaces on a single network, and that means that the interfaces are “bonded” into a single logical interface, which, then, has no routing issues whatsoever.

Having multiple network interfaces with IPs of the same network does not help with throughput, but does it increase redundancy? The simple answer is “no” – It does not. In Linux, when an interface is disconnected, it does not go down automatically (unlike Windows, for example), meaning that the routing table does not get updated with remaining interfaces. The traffic would still be targeted at that “dead” interface. Moreover – even if Linux did do that, this is a solution for a single type of a problem. What if someone changed the switch to have a different network VLAN on that particular (one of two or more) port? Link remains up, but communication doesn’t get anywhere.

These problems are more noticeable when the single network is an iSCSI network, and the system needs both multiple path access to the iSCSI storage, and redundancy becomes an issue, because iSCSI network is commonly – a critical network.

It is common to connect multiple iSCSI networks and not multiple ports on a single network, maintaining fabric-like separation of networks, and devices, where possible. However – this article will point at what to do if the network layout is not under your control and you need to place multiple network interfaces on the same physical network.

As mentioned before, the routing table will make sure that all your communication to that particular network go by the first routed network interface. However – the iSCSI daemon (open-iscsi) has the capability to bind to specific interfaces. To do so, you need to define an interface to iSCSI. Do so by running the command:

iscsiadm -m iface -I NIC_NAME –op=new

The name is user defined. I recommend using the same names used by the OS, like ‘eth5’ or ‘ib0’ or ‘eno1’ – to keep it simple. This is a declarative definition only. To actually bind the ‘iface’ to the real interface device, you need to run the following command:

iscsiadm -m iface -I NIC_NAME  –op=update -n iface.hwaddress -v 00:AA:BB:CC:DD:EE

Use the real interface MAC address. Then you can discover and login to targets with the defined interfaces only:

iscsiadm -m discovery -t st -p TARGET_IP -I NIC1 -I NIC2

iscsiadm -m node -L all -I NIC1 -I NIC2

According to the documentations, this will require setting the sysctl net.ipv4.conf.default.rp_filter to 0.

This should do the trick, so now multipathing should show the right amount of paths.