Re: Difficulties to get 1Gbps on be2net ethernet card

[Jean-Michel Hautbois <jhautbois@xxxxxxxxx>]

2012/5/30 Jean-Michel Hautbois <jhautbois@xxxxxxxxx>:
> 2012/5/30 Eric Dumazet <eric.dumazet@xxxxxxxxx>:
>> On Wed, 2012-05-30 at 08:51 +0200, Jean-Michel Hautbois wrote:
>>> 2012/5/30 Eric Dumazet <eric.dumazet@xxxxxxxxx>:
>>> > On Wed, 2012-05-30 at 08:28 +0200, Jean-Michel Hautbois wrote:
>>> >
>>> >> If this can help, setting tx queue length to 5000 seems to make the
>>> >> problem disappear.
>>> >
>>> > Then you should have drops at Qdisc layer (before your change to 5000)
>>> >
>>> > tc -s -d qdisc
>>> >
>>> >> I didn't specified it : MTU is 4096, UDP packets are 4000 bytes.
>>> >
>>>
>>> Yes :
>>> qdisc mq 0: dev eth1 root
>>>  Sent 5710049154383 bytes 1413544639 pkt (dropped 73078, overlimits 0
>>> requeues 281540)
>>>  backlog 0b 0p requeues 281540
>>>
>>> Why ? With a 2.6.26 kernel it works well with a tx queue length of 1000.
>>
>> If you send big bursts of packets, then you need a large enough queue.
>>
>> Maybe your kernel is now faster than before and queue fills faster, or
>> TX ring is smaller ?
>>
>> ethtool -g eth0
>>
>> Note that everybody try to reduce dumb queue sizes because of latencies.
>>
>
> TX ring is not the same :
> On 3.2 :
> $> ethtool -g eth1
> Ring parameters for eth1:
> Pre-set maximums:
> RX:             1024
> RX Mini:        0
> RX Jumbo:       0
> TX:             2048
> Current hardware settings:
> RX:             1024
> RX Mini:        0
> RX Jumbo:       0
> TX:             2048
>
>
> On 2.6.26 :
> $>ethtool -g eth1
> Ring parameters for eth1:
> Pre-set maximums:
> RX:             1024
> RX Mini:        0
> RX Jumbo:       0
> TX:             2048
> Current hardware settings:
> RX:             1003
> RX Mini:        0
> RX Jumbo:       0
> TX:             0
>
> I can't set TX ring using ethtool -G eth1 tx N : operation not supported
> I am not really impacted by latency, but the lower the better.
>
> JM

I used vmstat in order to see the differences between the two kernels.
The main difference is the number of interrupts per second.
I have an average of 87500 on 3.2 and 7500 on 2.6, 10 times lower !
I suspect the be2net driver to be the main cause, and I checkes the
/proc/interrupts file in order to be sure.

I have for eth1-tx on 2.6.26 about 2200 interrupts per second and 23000 on 3.2.
BTW, it is named eth1-q0 on 3.2 (and tx and rx are the same IRQ)
whereas there is eth1-rx0 and eth1-tx on 2.6.26.

JM
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html