Re: IOPS higher than expected on randwrite, direct=1 tests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

* Shawn Lewis <shawnlewis@xxxxxxxxxx> wrote:
> On Thu, Nov 11, 2010 at 8:22 AM, Sebastian Kayser <sebastian@xxxxxxxxxx> wrote:
> > Shawn Lewis sent me an email off-list and suggested that the disk itself
> > will still do write-caching (despite O_SYNC) unless this has been
> > explicitly disabled. I dug into the storage configuration and found an
> > option which sounded promising ("Drive Delayed Write", not to be confused
> > with the RAID controller write cache which was disabled from the start).
> > Disabled the delayed writes and the IOPS went down from ~240 to
> >
> > * ~100 for a filesystem test (size=100g out of 2TB)
> > * ~110 for a raw device test
> Did any of your tests use the full device or were you still setting
> the size= parameter? Set size=2T or get rid of the size= parameter
> entirely for the raw device test and you should see more like 70 or 80
> iops.

Shibby. :) Raw device tests without size= deliver an average of 71 IOPS,
thus the disk actually delivers less than what the theoretical datasheet
calculation [1] suggests. Or the non-available specs for our desktop range
harddisk are simply a bit inferior.

root@ubuntu-804-x64:~# ./fio --section=iscsi-raw patterns.fio
iscsi-raw: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0K/311K /s] [0/76 iops] [eta 00m:00s]
iscsi-raw: (groupid=0, jobs=1): err= 0: pid=5309
  write: io=515496KB, bw=293258B/s, iops=71, runt=1800008msec

> I believe part of the reason the gain is so huge with the drive cache
> is that you were also only using the outer diameter of the disk. If
> you use the full disk but enable drive cache I think you'll see about
> 140 or 150iops.

That's gold! Enabled the disk write cache again, tested without size=
and got 160-170 IOPS. Less caching effect than with size=100g.

So the moral of the story is "IOPS tests which don't utilize the full
device won't show its worst case IOPS performance and are pretty much
misleading (unless the application which will later be put onto the
device also only uses a similar subset of the device)"?!

I wonder how many times I saw someone - including myself - fire up
bonnie++ or iozone with a rather small test file compared to the full
disk size ... Thanks very much everyone! This was an (overdue) eye


To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at

[Home]     [Linux SCSI]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Photo]     [Yosemite News]    [Yosemite Photos]    [Video Projectors]     [Free Online Dating]     [Linux Kernel]     [Linux SCSI]     [XFree86]

Powered by Linux