Re: [ofa-general] iSer and Direct IO

Joe Landman <landman@xxxxxxxxxxxxxxxxxxxxxxx> · Thu, 15 May 2008 11:25:22 -0400

Cameron Harr wrote:

----

[root@test05 ~]# sgp_dd dio=1 if=/dev/zero of=/dev/fioa bs=512 bpt=2048 

count=16777216 time=1

This is only 8 GB of IO.  It is possible that (despite dio) you are 

caching.  Make the IO much larger than RAM.  Use a count of 128m or so.

time to transfer data was 5.556115 secs, 1546.03 MB/sec

[root@test05 ~]# sg_dd dio=1 if=/dev/zero of=/dev/fioa bs=512 bpt=2048 

count=16777216 time=1

time to transfer data: 5.565360 secs at 1543.46 MB/sec
[root@test05 ~]# dd oflag=direct if=/dev/zero of=/dev/fioa bs=1M count=8192
8589934592 bytes (8.6 GB) copied, 12.7761 seconds, 672 MB/s

We have found dd to be quite trustworthy with [oi]flag=direct.

----

Using iSer, with the small transfer chunks, sgp_dd has numbers that are in line 

with what I'd expect for DIO while sg_dd doesn't:

---------
sgp_dd:  200.64 MB/s
sg_dd:   735.42 MB/s
dd:     62.3 MB/s
--------

But for larger transfers (with 1M block transfers), both sgp_dd and sg_dd show 

well above what I think I can be getting:

-------
sgp_dd: 882.43
sg_dd:   819.89
dd:      731 MB/s #Which is still high, and which makes me suspect iSer

We had iSER bouncing from low 200s through 1000 MB/s during testing. 

Very hard to pin down good stable benchmark times.  This was a few 

months ago.

--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman@xxxxxxxxxxxxxxxxxxxxxxx
web  : http://www.scalableinformatics.com
       http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 866 888 3112
cell : +1 734 612 4615
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html