A low % of packets going out thru wrong interface

[Jorge Salamero Sanz <bencer@xxxxxxxxxxxxxx>]

Hi all,

I've a multigateway router doing load balancing. I've detected that a low % of 
packets are going out through the wrong interface.

To test this I've deployed the following scenario:

routerA 10.10.1.251  <=>  eth1 10.10.1.1/24
                                       router   eth3 192.168.1.1/24 <=> client
routerB 10.10.2.251  <=>  eth2 10.10.2.1/24


On the client I've a perl script doing wgets to different urls and on both 
routers I've tcpdump looking for wrong packets:

routerA:~# tcpdump -i eth1 -n src 10.10.2.1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
16:28:33.540895 IP 10.10.2.1.38572 > 192.168.100.254.53: S 
3734774276:3734774276(0) win 5840 <mss 1460,sackOK,timestamp 24681764 
0,nop,wscale 1>
16:28:57.541067 IP 10.10.2.1.38572 > 192.168.100.254.53: S 
3734774276:3734774276(0) win 5840 <mss 1460,sackOK,timestamp 24687764 
0,nop,wscale 1>
16:29:45.542669 IP 10.10.2.1.38572 > 192.168.100.254.53: S 
3734774276:3734774276(0) win 5840 <mss 1460,sackOK,timestamp 24699764 
0,nop,wscale 1>

routerB:~# tcpdump -i eth1 -n src 10.10.1.1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
16:38:25.020443 IP 10.10.1.1.44962 > 80.247.32.136.82: S 
3427612315:3427612315(0) win 5840 <mss 1460,sackOK,timestamp 24832042 
0,nop,wscale 1>
16:38:41.059188 IP 10.10.1.1.50276 > 212.175.239.74.90: S 
133519430:133519430(0) win 5840 <mss 1460,sackOK,timestamp 24836052 
0,nop,wscale 1>
16:39:29.056531 IP 10.10.1.1.50276 > 212.175.239.74.90: S 
133519430:133519430(0) win 5840 <mss 1460,sackOK,timestamp 24848052 
0,nop,wscale 1>
16:48:44.961472 IP 10.10.1.1.35885 > 12.19.68.8.8008: S 250197835:250197835(0) 
win 5840 <mss 1460,sackOK,timestamp 24987036 0,nop,wscale 1>

The load balacing router is hardy 2.6.24 with the following setup:

# ip rule ls
0:      from all lookup local 
32761:  from all lookup main 
32762:  from 10.10.1.1 lookup 2 
32763:  from all fwmark 0x2 lookup 2 
32764:  from 10.10.2.1 lookup 1 
32765:  from all fwmark 0x1 lookup 1 
32766:  from all lookup main 
32767:  from all lookup default

# ip route ls table main
10.6.7.0/24 dev eth0  proto kernel  scope link  src 10.6.7.12 
10.10.2.0/24 dev eth2  proto kernel  scope link  src 10.10.2.1 
192.168.1.0/24 dev eth3  proto kernel  scope link  src 192.168.1.1 
10.10.1.0/24 dev eth1  proto kernel  scope link  src 10.10.1.1 
# ip route ls table 1
default via 10.10.2.251 dev eth2 
# ip route ls table 2
default via 10.10.1.251 dev eth1 
# ip route ls table default
default 
        nexthop via 10.10.2.251  dev eth2 weight 1
        nexthop via 10.10.1.251  dev eth1 weight 1

iptables rules just have MASQUERADING, but I've tried dropping invalid 
packets, SNAT, etc with the same results. Nothing on mangle but if I setup 
rules for policy routing using marks I also can observer same behavior.

Do you know how could I debug this issue or what could be wrong ? Clients can 
browse when packets of their conections go out using the wrong iface.

Thanks in advance!
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html