Multipath route

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



Hello,

Some time ago we were faced with a problem with multiple network interfaces in the same subnetwork. We would like to use several NIC's with IP addresses from the same subnetwork with link redundancy functionality.

For example - we've got three NIC's at some host and each of them have its separate IP addresses:

eth0: 11.0.0.1
eth1: 11.0.0.2
eth2: 11.0.0.3

from 11.0.0.0/24 subnet.

Now we would like to define some route for all of those interfaces like that:

# ip route add 11.0.0.0/24 nexthop dev eth0 nexthop dev eth1 nexthop dev eth2

As far as I know there is only one algorithm for handling multipath routes in 2.6.33 kernel. It doesn't care about physical link status. It uses randomization combined with weights assigned for interfaces and provides some kind of load balancing.

But we are not interested in load balancing. We are rather interested in physical link redundancy.

Here there is how such solution can look like:
- we can define bunch of interfaces as the "nexthop dev" parameters and assign unique priority to each of them - we can use brand new algorithm which allows to select the highest priority interface (with physical link up) and use it for communication.

For example if we have such a configuration:

eth0: prio 1 (highest priority) link up
eth1: prio 2                    link up
eth2: prio 3 (lowest priority)  link up

then the eth0 will be *the only interface* to reach other hosts from 11.0.0.0/24 subnet. When the eth0 link becomes down - the eth1 will be used. Next if eth1 also becomes down - the eth2 will be used.
But if now eth0 becomes up - it will be selected as the default interface.

That is our imagination how it should work.


For testing purpose we've prepared some very simple patch for 2.6.33 kernel with this functionality.
It is available in the attachment.

It requires a few additional commands while configuring, for example:

# ip route add 11.0.0.0/24 nexthop dev eth0 nexthop dev eth1 nexthop dev eth2
# echo 1 > /proc/sys/net/ipv4/route/nexthop_alg
# ip rule add from 11.0.0.1 table 1
# ip route add src 11.0.0.1 dev eth0 table 1
# ip rule add from 11.0.0.2 table 2
# ip route add src 11.0.0.2 dev eth1 table 2
# ip rule add from 11.0.0.3 table 3
# ip route add src 11.0.0.3 dev eth2 table 3

In example above the eth0 has the highest priority and the eth2 has the lowest one. With "echo" we're selecting the algorithm "1" which is the implementation of algorithm described above. Value "0" switches to the original kernel algorithm (this is also the default value).

Depending on the physical link state of eth0, eth1 and eth2 it should switch all of new connections from this host addressed to the 11.0.0.0/24 between eth0, eth1 and eth2 interfaces. It concerns only the connections initiated from this host to 11.0.0.0/24 subnetwork. The behavior of this algorithm can be observed using tcpdump - while unplugging eth0 the whole network traffic to 11.0.0.0/24 should switch from eth0 to eth1.

We know that this solution is far from being perfect. One of the drawbacks is that it loses old TCP/IP connections established previously on the link which has just gone down.

For example - we've established telnet connection with another host (11.0.0.10) while the eth0 was selected as the highest priority interface. This connection is working fine for some time and then the eth0 physical link for some reasons becomes down. At that moment the new algorithm from this patch selects next interface with physical link up - and it will be eth1 in this case. We are able to start new telnet connection with 11.0.0.10 (this time it will goes through eth1), but we will not be able to use the old telnet connection we established previously on eth0 interface. So - all the connections established earlier via eth0 cannot be restored as long as eth0 will be down.

It seems that this is not a fully-featured link redundancy solution, but for now this is acceptable for us.

Keeping in mind the description above we would like to ask if there is any other solution currently available and ready to use?


WK


--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Netdev]     [Ethernet Bridging]     [Linux 802.1Q VLAN]     [Linux Wireless]     [Kernel Newbies]     [Security]     [Linux for Hams]     [Netfilter]     [Git]     [Bugtraq]     [Photo]     [Yosemite News and Information]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux PCI]     [Linux Admin]     [Samba]     [Video 4 Linux]