Re: vxlan/veth performance issues on net.git + latest kernels

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Fri, Dec 6, 2013 at 3:36 PM, Eric Dumazet <eric.dumazet@xxxxxxxxx> wrote:
> On Fri, 2013-12-06 at 11:06 +0200, Or Gerlitz wrote:
>> On Wed, Dec 4, 2013 at 3:23 AM, Eric Dumazet <eric.dumazet@xxxxxxxxx> wrote:
>> > skb->dev is in the first cache line, where we access skb->next anyway.
>> > I could use skb->cb[] like the following patch :
>>
>> Hi Eric, I applied on the net tree the patch you posted yesterday
>> "net: introduce dev_consume_skb_any()" along with the network drivers
>> part of this patch, unless I got it wrong, I assume both pieces are
>> needed?
>>
>> So I re-run the vxlan/veth test that we suspect goes through packet drops on TX.
>>
>> With the patches applied I have almost no samples of that event
>>
>> $ ./perf report -i perf.data
>
> How did you get this perf.data file ? There are a few drops.

Using the command you suggested on the active side while running
traffic (iperf tcp)

$ ./perf record -e skb:kfree_skb -g -a sleep 10

>> Samples: 89  of event 'skb:kfree_skb', Event count (approx.): 89
>> +  39.33%  ksoftirqd/2  [kernel.kallsyms]  [k] net_tx_action
>> +  28.09%      swapper  [kernel.kallsyms]  [k] net_tx_action
>> +  28.09%         sshd  [kernel.kallsyms]  [k] net_tx_action
>> +   2.25%      swapper  [kernel.kallsyms]  [k] kfree_skb
>> +   1.12%  kworker/2:2  [kernel.kallsyms]  [k] net_tx_action
>> +   1.12%        iperf  [kernel.kallsyms]  [k] net_tx_action

>>  ./perf report -i perf.data --sort dso,symbol
>> Samples: 89  of event 'skb:kfree_skb', Event count (approx.): 89
>> +  97.75%  [kernel.kallsyms]  [k] net_tx_action
>> +   2.25%  [kernel.kallsyms]  [k] kfree_skb
>> --

> OK, this means your driver drops few packets in its ndo_start_xmit()
> handler.
>
> Could you give us "ifconfig -a" reports as I already asked  ?

will do tomorrow while infront of the setup.

When I provided you the info last time
http://marc.info/?l=linux-netdev&m=138610891121531&w=2 it included the
ifconfig -a output and no drops were seen there


>
> You could temporary change the dev_kfree_skb_any() in mlx4_en_xmit()
> to call kfree_skb(skb) instead, to get  a stack trace (perf record -a -g
> -e skb:kfree_skb sleep 20 ; perf report)

yes, tomorrow

>
> diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
> index f54ebd5a1702..53130f27dec0 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
> @@ -873,7 +873,7 @@ tx_drop_unmap:
>         }
>
>  tx_drop:
> -       dev_kfree_skb_any(skb);
> +       kfree_skb(skb);
>         priv->stats.tx_dropped++;
>         return NETDEV_TX_OK;
>  }
>
>
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Kernel Discussion]     [TCP Instrumentation]     [Ethernet Bridging]     [Linux Wireless Networking]     [Linux WPAN Networking]     [Linux Host AP]     [Linux WPAN Networking]     [Linux Bluetooth Networking]     [Linux ATH6KL Networking]     [Linux Networking Users]     [Linux Coverity]     [VLAN]     [Git]     [IETF Annouce]     [Linux Assembly]     [Security]     [Bugtraq]     [Yosemite Information]     [MIPS Linux]     [ARM Linux Kernel]     [ARM Linux]     [Linux Virtualization]     [Linux IDE]     [Linux RAID]     [Linux SCSI]