Re: Supermicro X8DTH-6: Only ~250MiB/s from RAID<->RAID over 10GbE?

[Justin Piszcz <jpiszcz@xxxxxxxxxxxxxxx>]

On Sat, 5 Feb 2011, Dr. David Alan Gilbert wrote:

> * Justin Piszcz (jpiszcz@xxxxxxxxxxxxxxx) wrote:
> > On Sat, 5 Feb 2011, Emmanuel Florac wrote:
> >
> > > Le Sat, 5 Feb 2011 14:35:52 -0500 (EST) vous écriviez:
> >
> > To respond to everyone:
> >
> > > Did you try launching 4 simultaneous cp operations over nfs to get to
> > > 1.2 GB/s?
> > >  I've witnessed single stream copy performance with Samba being less than
> > > maximum due to Samba limitations.  Running multiple copy ops in parallel then
> > > usually saturates the pipe.
> >
> >
> > I tried 4 simultaenous cp's and there was little change, 250-320MiB/s.
>
> So I think you've said network benchmarks are OK, disc benchmarks are OK, but
> a copy over the network is slow.
>
> What happens if you run a disc benchmark at the same time as a network benchmark;
> even though the two aren't related? Can you keep the disc busy writing even when
> the network is being pushed?

Hi,

Still OK when reading or writing & iperf test, so it does not appear to be
I/O or backplane bound.  I also put both cards on the same CPU (lane-wise)
and it made no difference.

Device eth0 [10.0.1.4] (1/1):
================================================================================
Incoming:
                                                  ###
                                                  ###
                                                  ###
                                                  ###
                                                  ###
                                                  ###  Curr: 1188.85 MByte/s
                                                  ###  Avg: 680.11 MByte/s
                                                  ###  Min: 0.00 MByte/s
                                                  ###  Max: 1188.88 MByte/s
                                                  ###  Ttl: 18.32 GByte
Outgoing: